University of Chicago Center for International Studies
and Argonne National Laboratory
Threat Anticipation: Social Science Methods and Models [Apri 7-9, 2005]
- Investigation of social science disciplines, research, theory, and formal models applicable to understanding and modeling threats
- Presentation of models developed for threat anticipation and discussion of future needs
- Working group discussion of modeling issues and emerging threat areas
The workshop began on Thursday, April 7, 2005 at the University of Chicago Hyde Park campus with a panel discussion on social science and modeling, followed by a reception at the UC International House. The main workshop commenced on Friday, April 8, 2005 at the University of Chicago Gleacher Center on North Michigan Avenue. Friday’s program consisted of plenary sessions on the DTRA/ASCO models, ongoing social science research related to threat anticipation, and issues in model validation and usage. On Saturday, April 9, the participants attended parallel sessions with working groups discussing various areas related to modeling threats (AM) and social science research issues relating to threats (PM).
The Workshop Sessions
“History, Culture, and Computation: Tough Questions about the Relationship between Models and Human Behavior”
The panelists were asked by the conference organizers to address a number of questions, including:
- What is the role and current state of formal models and computational models in the social sciences?
- What are examples of modeling gone wrong and common errors to be avoided?
- What are the challenges and prospects for developing integrative formal and computational models that cross disciplines within the social sciences?
- What are the prospects of developing formal and computational models in the social sciences that can contribute to policy making?
In addition to discussing these issues, Yona Rubinstein presented a brief summary of his own social science modeling. His research (a collaboration with Gary Becker, of the University of Chicago’s Economics Department) uses rational choice models to explain why low-probability events such as the likelihood to be harmed by terrorism nevertheless affect behavior.
Related Material: "The Rational Response to Terrorism," Economist, 376, 72 (2005).
Yona Rubinstein and Gary Becker "Fear and the Response to Terrorism: an Economic Analysis" Working Paper, August 2004.
“Welcome” - Kathleen Morrison, Joint Threat Anticipation Center, Center for International Studies, University of Chicago; Charles Macal, Joint Threat Anticipation Center, Center for Complex Adaptive Agent Systems Simulation, Argonne National Laboratory
Morrison began with a discussion on the nature of threat anticipation. For the purposes of the Joint Threat Anticipation Center, threats to the nation’s defense and security, as well as to its citizens were key components of its scope. However, the nature of such threats is diverse and requires a diverse knowledge base, including domestic and international research at a variety of levels and encompassing a number of disciplines. The common thread to this complex approach is a better understanding of human behavior, particularly at the group level.
Morrison also discussed the nature of the collaboration between Argonne National Laboratory and the University of Chicago. She highlighted the many aspects of the campus entity she directs, the Center for International Studies (CIS). Within CIS, there are a number of programs, including the Human Rights Program, the Transnationalism Project, and the Program for the Study of Violence, Conflict, and Security. CIS also coordinates with four regionally focused area studies centers, and can thus share with Argonne a broad base of social science and area knowledge.
Macal followed Morrison’s presentation with a summary of Argonne projects currently supported by DTRA/ASCO. The first of these is the Center for Complex Adaptive Agent Systems Simulation, which focuses on model analysis, model evaluation, and the modeling of culture. Through Argonne, DTRA/ASCO also funds the Chicago Project on Suicide Terrorism. With these projects and those expected to receive support during the 2005-06 FY, the Joint Threat Anticipation Center will work with DTRA/ASCO to focus on threat anticipation modeling by concentrating on inequality, poverty, and terrorism.
Macal concluded by identifying the goals of the first phase of the Joint Threat Anticipation Center:
- to identify frameworks/models for integrating cultural knowledge in meaningful ways for policy
- to develop a broad base of knowledge relevant for future security threats
- to evaluate ASCO models from theoretical and methodological underpinnings
- to establish an ongoing dialogue between social scientists and computational modelers.
Hamon provided a brief history of the Advanced Systems Concepts Office of DTRA. Since its creation in 1999, the office has commissioned or supported thirty to forty studies a year. Prior to 9/11, ASCO was charged with anticipating threats to national security before they reached crisis level. After the failure to anticipate the September 11 th attacks or the lingering threat of terrorism that preceded it, it was clear that new anticipation methods were critical.
ASCO’s early projects focused on nuclear strategy in keeping with DTRA’s traditional area of interest. Since 2001, ASCO has sought new projects which reflect the changing nature of threats to the United States. In large part, this has involved projects that bring the computational and social sciences together to model the challenges of the 21 st century.
The new initiative has found success in the development of projects like JTAC, but a number of tests remain. Most importantly, an effective method for peer review must be established, and the models must be verified and validated – a process that is particularly difficult when the nature of the research fall outside of the boundaries of the traditional disciplines. Hamon expressed hope that the University of Chicago can meet the interdisciplinary needs of these new models and play an important role in the development and exploration of new modeling methods.
Younger began with brief introductory remarks on his own experience as a physicist with social science modeling in the late 1980s, concentrating on ethical situations at a societal level. Younger then went on to discuss a series of studies released in June 2001 that hoped to reframe understandings of mass violence in genocide. Those studies forced a reevaluation at DTRA, as the agency realized that the ability to wage mass violence has moved from nation-states to non-state actors. Simply put, parties existing outside of the realm of international politics now had the capacity to kill not just tens or hundreds of people, but hundreds of thousands.
According to Younger, the United States now finds itself at the end of a post-Cold War era where the assumptions of the 1990s may no longer apply. In this new geo-strategic environment, it is difficult to understand both the nature of terrorism and the motivation of terrorists. Simulations of social systems can be useful tools in this new context, and modeling and simulation have attracted the interest of a number of government agencies.
During the Cold War, the standoff between the two superpowers created a situation in which policy and decision makers did not know and did not understand their counterparts. American identity was largely formed in opposition to the Soviets, which found the United States without clearly defined goals in the post-Cold War period. Younger noted that the post-Cold War period is over and that American identity and goals are now tied to its identity as a democracy and to the spread of democracy throughout the world.
The first test in this new period was in the occupation of Iraq. The Bush Administration found that Iraqis did not embrace democracy as quickly as had been hoped and resisted the U.S. occupation. Although the United States cannot expect to suffer defeat given conventional military superiority, the war in Iraq has been costly ($300 billion) and an effective defense against suicide bombers still has not been established.
There exists a need to look at the causes of violence. From Younger’s point of view, the answers clearly lie in the social sciences, and this is where the models should come from. However, it is difficult to validate models of society and of societal phenomena. He found promising current research on violence in egalitarian societies and on the restrictions on violence found in different societies.
Younger concluded by stressing the importance of shifting the focus in national security studies from the physical sciences towards the social sciences in order to better understand the people behind the threats.
Presentations on Models Session
MacKerrow’s work on the root causes of terrorism and insurgencies was undertaken with the hope of establishing a system of rigorous, repeatable, and controlled experiments that had a clear end state matrix suitable for evaluation by outsiders.
The Threat Anticipation Model was designed to address a number of problems in more traditional terrorism/insurgency models. In particular, MacKerrow noted that models that concentrate on socio-economic indicators (for example Gurr’s relative deprivation model) show little correlation with the conditions of known Islamist insurgencies in Algeria, Egypt, Jordan, Morocco, and Tunisia. Thus, MacKerrow set out to examine other variables, including economic ones, which might better explain the occurrence of insurgency.
The Threat Anticipation Model employs a political process approach, which assumes that exclusion from political and institutional processes create defined reactions. This approach requires an understanding of the region in question as well as the input of anthropologists who examine the agents, their objectives, and the means they use to achieve those objectives.
Within the Threat Anticipation model is a micro-model known as Grievance Simulation, which looks at inputs such as political participation, political/religious alignment, perceived hardship, missed expectations, and social welfare. Grievance Simulation also considers agents’ feelings toward other identity groups and “allegiance distribution,” or the agents’ position on a particular issue. A significant challenge lies in attempts to operationalize the changing level of grievance – in itself a function of comparison with other groups.
There are two case studies at present: (1) a simple model of tribal behavior in Pakistan and why particular groups abandon traditions; and (2) an examination of Hezbollah which allows an examination of the stated objectives of terrorists and how agents meet or fail to meet those objectives.
MacKerrow acknowledged two challenges with the Threat Anticipation Model. The first is that specific problems and questions require a specific model, given the complex nature of the system. Second, validation of social science models is difficult and requires a number of approaches including experimental economics, software verification, and uncertainty analysis among others.
Related material: Edward MacKerrow “Understanding Why – Dissecting radical Islamist terrorism with agent-based simulation,” Los Alamos Science, Number 28, 2003
At its core, IndaSea’s Cultural Simulation Model (CSM) uses data inputs from daily news in order to understand culturally-specific points of view, based on an assumption that asymmetric threats such as terrorism are intertwined with cultural, social, economic, and political conditions. IndaSea’s work represents an effort to construct an inclusive model based on these multiple perspectives, including bias.
The construction of the CSM has gone through three phases. In the first, a prototype was constructed to test the software with minimal inputs. The second phase found an application in Islamist movements in the Middle East, and the third will focus on terrorism and Islamist groups in Indonesia.
The system is designed around a perception-based metaphor to understand actors’ responses. Context is an essential aspect of the design, as is the fact that the data can be in multiple languages and focuses on culturally-specific rhetoric. Every word in the news sources is indexed and forms its own connection. At present, the CSM relies on 1400 words to explain the behavior of actors. The words are then linked to images to allow the end user to visualize the dominant words.
The CSM is currently using three different groups as actors, Jemaah Islamiya, Indonesian jihadis, and a broadly defined group of “Salafi Muslims.”
Related material: IndaSea White Paper, March 2005
The goal of Netbreaker is to elucidate possible terrorist networks before they are able to act. In introducing the project, North noted that while there is some degree of chance in formation of networks, they are not completely accidental – there are discernible rules, structures and patterns to the networks. The primary question is whether one can infer large social network structures from a relatively small data set. If this is possible, three additional questions can be asked:
- What structures are likely to produce the observations?
- What questions should be asked?
- Can uncertainty be quantified?
The Netbreaker model uses agent-based social modeling to find possible terrorist networks. As a tool, the goal of Netbreaker is to reduce surprise by providing and quantifying possibilities, not to determine by itself what possibility is correct. It is up to the end user to determine which is correct, thus the human analyst is the principal part of the investigative process.
Netbreaker functions in two ways. First, it acts as a simulation to determine the capabilities of a group: how it can disseminate ideas or opinions, distribute and spend money, etc. Secondly, Netbreaker provides a number of alternative networks for the analyst to consider, presenting varying shapes for a network, identifying interactions between agents, and identifying what interaction may mean for the overall likelihood of a threat. Functioning in this way, Netbreaker essentially generates questions, tracks the construction of a network, and makes a quantitative assessment of interactions. As the analyst provides the model with more data, a smaller number of possibilities come to light leading to an increasingly more accurate model of the actual network.
Related materials: Michael North, Charles Macal, and Jerry Vos “Terrorist Simulation with NetBreaker”
Goldsmith’s research involves the development of Statistical Machine Translation (SMT), which has become more plausible and successful as computing power has greatly increased in the last decade. SMT takes a text in an unknown language and develops a basic grammatical analysis of the language, with the major emphasis on morphology (word-structure). Goldsmith’s hope is that the system will hasten the development of fully automated machine translation (MT) for the less-studied languages.
In his comments on the morning’s models, particularly on the IndaSea model, Goldsmith noted that training data is key and that the models generated are probabilistic. He also noted that for testing and evaluation, the evaluator should make predictions by studying other, closer networks, and not be limited to terrorism. Finally, Goldsmith expressed the opinion that the greatest impetus to successful modeling is to have groups that compete with one another.
Thomas Howe made a number of suggestions for social science modeling. First, he stressed that agents should work in an appropriate context. Second, he emphasized the need for rigorous verification and validation, especially because the models at hand deal with threat assessment. This would become an important theme in the discussions that followed.
Howe also called for the modelers to sustain heuristics, noting that there was a significant language divide between domain experts and modelers. He suggested that it was entirely possible for all involved to develop systems of domain-specific languages.
Finally, Howe said that modelers should be very open and transparent as to the goals and assumptions behind their models.
Patil followed Howe’s presentation with a number of recommendations of his own. Most importantly, he explained that analysts and end users of models should be kept in the loop during the development process. The analyst can play a critical role in obtaining and distributing data, assessing the current state, and projecting and forecasting threats.
Patil also discussed the variety and types of models that were possible in social science, especially when one considers the scales within the various disciplines – psychology focuses on the relatively short term, while sociology concentrates on the long term. Ideally, suggests Patil, models should incorporate a range of scales.
Patil’s final points focused on verification and validation, suggesting that validation should focus on utility and forecasting.
A number of questions were raised in the lively discussion that followed:
- What are the incentives for modeling community to share code?
- Is the natural language process open source or open algorithm?
- How can work be facilitated between the academic, commercial, and government communities of modelers?
- In regard to validation – are we making the correct decisions based on uncertainty?
- What is the gold standard for validation?
- There was recognition that peer review is essential for verification and validation, but what method of communication is most efficient for review and how can (or should) commercial interests be protected?
Social Science: Applicable Theory and Models
The Chicago Project on Suicide Terrorism is focused on understanding the causes, conduct, and consequences of suicide terrorism. This project aims to: (1) establish and maintain an intensive, focused, collaborative, and multidimensional approach to analyzing suicide terrorism events; (2) create a comprehensive cross-regional knowledge base for analysis of suicide terrorism; (3) conduct research and developmental activities on the strategic, economic, religious, and cultural factors that influence the causes, conduct, and consequences of suicide terrorism and responses to it; and (4) to train people in fields relevant to the analysis of terrorism.
As a first step, Pape constructed a complete worldwide knowledge base of suicide terrorist attacks, martyr videos, and terrorist interviews, 1980 to 2001, and is working on adding data from the last four years. The database will also be expanded to include systematic information on the demographic and biographical characteristics of suicide attackers using native language sources. All martyr videos from around the world will be collected, translated, and made available to researchers, modelers, and other experts for analysis. Although such videos are widely publicized in the regions associated with the terrorists, there is now no accessible archive of martyr videos available for analysis in the United States, Europe, or other major centers for the study of terrorism.
Pape and his team of graduate students and other area experts have also spent considerable time on strategic analysis. Even if individuals are motivated by religion, economic and/or cultural factors, terrorist organizations are often induced by rational incentives in the composition of a suicide terrorist campaign, selection of targets, and social basis of support for their activities. Research that evaluates the strategic consequences of external events and internal dynamics on suicide terrorist campaigns and their tactical, operational, and political effects will identify likely tipping points that push terrorist groups and networks toward radically altering their objectives and capabilities.
Related Materials: Robert Pape "The Strategic Logic of Suicide Terrorism" American Political Science Review, 93, 343-361 (2003)
Discussant: Salim Yaqub, UC/History
Yaqub began by commenting on Pape’s article on his research, published in the Journal of the American Political Science Association. In the article, Pape suggests that Israel surrendered territory to the Palestinians as a result of terrorist attacks in the mid-1990s. Yaqub suggested that it was not a binary relationship between Israel and a single Palestinian entity, and that perhaps the Palestinian suicide attacks could be better understood by examining the status of the Palestinian authority versus that of the extremists, suggesting a triangular model. In Yaqub’s analysis suicide terrorism worked when the scale of violence was low and when the perceived distance between the terrorists and the Palestinian authority was great. This would then explain a number of moments in the 1990s when Israel made concessions to the Palestinians. Yaqub also noted that after 2000 and the beginning of the current intifada, the split between the two factions dissolved and that this would explain the lack of territorial concessions since then.
A number of questions were raised by participants and answered and debated by Pape and Yaqub:
- Is suicide terrorism dependent upon community support?
- What about unsuccessful attacks and what factors turn people away?
- Is there a supply and demand problem for suicide attackers? There is a supply of less educated potential attackers but the demand for educated attackers is higher.
- Does Pape count as terrorist acts those that are carried out against military targets? This point led to a lively discussion about what various assumptions on this issue would mean for Pape’s conclusions.
Townsend’s work studies in detail the social, economic and religious context associated with violence and terrorism. Based on six years of survey in northeastern and central Thailand and two years of field research in Muslim-majority provinces in the south, as well as a cross-sectional survey of 250 households in provinces bordering on Malaysia, Townsend has constructed a GIS-based research archive of key economic and social indicators.
The research is designed to allow an evaluation of both informal and formal financial and social systems. There is a planned expansion of the GIS archive and extensive survey techniques developed for other parts of the country to be extended to the Southern provinces. Annual resurveys will be continued, along with an intensive micro-economic and ethnographic survey (already in progress elsewhere). Formal instruments are administered in monthly interviews, gathering data on consumption, inputs and output, crops, revenues and expenses in business, labor supply, assets purchases and sales, remittances and gifts, borrowing and lending, and grain and other inventory. New surveys are planned to cover health and education, including an assessment of religious and weekend training in government vs. Islamic schools.
More generally, Townsend’s research aspires to demonstrate how to link economic models of growth with microeconomic foundations in such a way as to understand the forces which appear to be correlated with violence and the potential for terrorism – growth in regions or countries which is not uniform, inequality in the distribution of income/wealth, disparity in social services, and bargaining for state resources. Townsend sees a useful collaboration between economists, anthropologists, computer modeling and environmental/resource assessments in order to make predictions at various levels of disaggregation.
Discussant: Pamela Sydelko, Argonne National Laboratory
Sydelko began her comments by noting that Townsend’s work was evidence of the utility of GIS in modeling. The technology was initially seen as impossibly complex, indicating the rapid advances in computing power and technology in the last decade.
Sydelko expressed concerns in three areas. First, she noted that validation of Townsend’s data was necessary, but to do so appropriately would be cost-prohibitive. Second, she noted that there were some limitations to this type of work, the availability of time to conduct the surveys and access to the important populations. Finally, she noted that in this type of research, a balance must be struck between the development of a useful model and the amount of detail that is possible but may not be productive in the model.
Questions raised and discussed by workshop participants included:
- What are the time horizons?
- How can poverty and inequality be weighted with cultural issues in examining motivations for violence?
- How can Townsend’s data be linked with that on the suicide attackers collected by Pape?
- Is there a need to study environmental data to investigate inequality in that area?
Computational Social Science: Framing Applicable Theory, Approaches, and Methods
Sallach began by defining the nature of a ‘threat’ (“the intent of an adversary pursuing a goal with respect to a target”), and considering the changing nature of threat dynamics in history. He noted that intent is not directly observable, and this poses a epistemological challenge for social science.
While empirical investigations make a vital contribution to scientific investigations, empiricism has limitations as well. Specifically, the only effective way to model what cannot be observed is through the development of effective theories. These ‘beables’ include the generation and evolution of intent, private communications and covert actions, and dense interaction patterns.
Social processes manifest bounded rationality, fractal perspectives and multiple simultaneous motives. In the case of extreme threats, the latter may include: fear, status seeking, revenge, conformity, material gain, in-group solidarity, and the lust for power. Of course, none of these motives, or how they combine, is directly observable.
An effective social theory will need to consider the role of emotion in the generation of social action, the way that actors coordinate in defining the situations they face, what actions are possible in particular circumstances, and how various types of cultures shape what groups value &/or regard as legitimate. A variety of contributions to social theory hold potentially valuable insights into such evanescent factors, including models of intentionality as an attractor system (Juarrero), the role of symmetric (but ambiguous) status maintenance (Gould), and recognition theory (Honneth).
It is possible that a consilient approach to social theory provides a framework that allows such contributions to be integrated. A robust and insightful theoretical framework will then allow us to more effectively address the unfolding of threats in history and, especially, in the contemporary world.
Related Material: David Sallach, "Interpretive Agents: Identifying Principles, Designing Mechanisms" from Proceedings of Agent 2003: Challenges in Social Simulation, October 2003.
David Sallach, "Social Theory and Agent Architectures," Social Science Computer Review, 21, 179-195 (2003).
Turnley followed Macal with an emphatic discussion of the central role of validation, stating simply that that the “goodness” of a model could not be assessed in any other way.
Turnley noted that computational models represent only part of a system, and it is important for the target user to understand which part. The approach of the modeler constrains the choice of data, and it seemed to remain an open question whether there is “interpretation free” social data and if the data used must be measurable and observable. However, theoretical frames help address these issues in defining the model’s purpose and informing the data that is ultimately chosen. If the end user understands the theoretical frame, s/he will be able to use the model more effectively. The challenge for the modeler is to translate a narrative into a computationally malleable structure. This forces assumptions to be made explicit, while also forcing the modeler to use data that is compatible with the computational capabilities of the model.
Evaluation of a model’s output falls into two principal concerns. First, the user must ascertain whether the model is an accurate representation of the system of concern. The challenge in this is to determine how to validate human social systems and a need to tie directly to the conceptual model and data. The second concern examines the extent to which the model addresses the problem at hand. The person doing the validation must establish if the appropriate frame was used, and how using a different one would affect the model. It also must be established that the correct data was used.
For validation, Turnley noted that one should concern oneself with what social reality is being represented. This is where the relevant social science plays an important role: establishing the nature of the state, the boundaries of the system, examining observer bias, and providing confidence for the quality of the data. The social scientist can also examine the selection logic and work with the modeler to determine how the computational format constrains data type and selection.
Zonis focused on non-terror threats to the United States, mainly economic in nature. He began by looking at the global economy as a whole and the interactions between rich countries, emerging markets, and failed states. The latter represent a particular problem as they tend to be oil-exporting countries with rental income, and the threat from them – places like Iran – is growing. However, the rich countries must continue to interact with the failed states because world petroleum demand is increasing, putting us in risky relationships with both the currently failed states and those which may create future problems, such as Russia.
The second major threat Zonis discussed is the decline of the dollar and the United States’ large deficits. In Zonis’ presentation, the US has become dependent on foreign investors, and yet the money brought in goes for material goods, not to increase US productivity. Worse, the current account deficit is increasingly owed to fewer nations – 25% is currently from China alone. Zonis believes that the threat of a money crisis is looming, and that it may not be a “soft landing.” To avoid a precipitous fall, other economies must grow enough to start buying more US exports. The US could also increase the value of the dollar if it was willing to reduce the deficit. Zonis saw neither of these conditions coming to pass in the near future, and thus the threat of a crisis will only grow.
Zonis concluded with some suggestions on managing failed states. He pointed out that currently, the United States spends very little money as a share of GDP on aid, and yet nation-building through economics is more likely to bring success than military solutions.