This is actually very exciting.
http://www.dhs.gov/xnews/releases/pr_1232576802004.shtm
It looks like the Secretary is seeking to put a renewed emphasis on formal risk analysis. The previous administration talked alot about risk management, but did not seem to embrace formal risk analysis.
Thursday, January 22, 2009
Wednesday, January 21, 2009
Black Swans
Nassim Taleb's latest book about "black swans" has garnered a good deal of attention within the homeland security risk community recently. Unfortunately, I have seen some leaders (who probably have not read the book) used the black swan argument as a reason not to do risk analysis. The NYT Sunday Magazine recently provided a good analysis of a similar type debate that is going on in the financial community (http://www.nytimes.com/2009/01/04/magazine/04risk-t.html).
The arguments against doing homeland security risk management basically go along these lines: terrorists are dynamic and unpredictable. Traditional risk analysis doesn't work because we can't depend on historical data to predict future attacks. Further, traditional risk analysis used in the actuarial circles doesn't account for "threat-shifting." In other words, if you implement a risk mitigation against flooding, you dont have to worry about mother nature deciding to change tactics and implement a different approach to "attack" and infrastructure or population. With terrorists, you do have to assume that they will adapt to countermeasures and change tactics over time.
I have seen folks within DHS basically use this type of reasoning to say that it is pointless to do risk analysis for terrorism issues. Why try to calculate risks when we will always be faced with black swans?
There are a few ways to answer this question. Most simply, if we dont do risk analysis, then everything is a black swan. A systematic risk management program should thoroughly evaluate all possible risks, and develop mitigation plans/measures that (1) are cost effective in reducing total risk across the spanning set of scenarios and/or (2) focus on the highest risks. Yes, there is always the chance that a risk will be overlooked or missed, but it doesnt make sense scrap the whole process because of that. Further, if you are looking for mitigation measures that are most cost effective across the spanning set of scenarios, it is highly likely that will be effective against scenarios that you couldnt come up with.
Another way to answer this question is by looking at the definition of black swan itself. Taleb developed the concept of black swans largely based on his experiences with VaR models. These models do not accurately take into account fat-tailed events (i.e., low probability, high consequence events). However, homeland security risk analyses are typically focused exclusively on fat-tailed events. When defining scenarios to be analyzed in a homeland security risk analysis, we typically only include these low probability, high consequence events. There is some work now to begin looking at how we can combine high probability, low consequence events (e.g., illegal immigration) into the same risk analyses as low probability, high consequence events, but typically these scenarios are analyzed seperately. In a way, we are already addressing the concerns of Taleb in our risk analyses.
The arguments against doing homeland security risk management basically go along these lines: terrorists are dynamic and unpredictable. Traditional risk analysis doesn't work because we can't depend on historical data to predict future attacks. Further, traditional risk analysis used in the actuarial circles doesn't account for "threat-shifting." In other words, if you implement a risk mitigation against flooding, you dont have to worry about mother nature deciding to change tactics and implement a different approach to "attack" and infrastructure or population. With terrorists, you do have to assume that they will adapt to countermeasures and change tactics over time.
I have seen folks within DHS basically use this type of reasoning to say that it is pointless to do risk analysis for terrorism issues. Why try to calculate risks when we will always be faced with black swans?
There are a few ways to answer this question. Most simply, if we dont do risk analysis, then everything is a black swan. A systematic risk management program should thoroughly evaluate all possible risks, and develop mitigation plans/measures that (1) are cost effective in reducing total risk across the spanning set of scenarios and/or (2) focus on the highest risks. Yes, there is always the chance that a risk will be overlooked or missed, but it doesnt make sense scrap the whole process because of that. Further, if you are looking for mitigation measures that are most cost effective across the spanning set of scenarios, it is highly likely that will be effective against scenarios that you couldnt come up with.
Another way to answer this question is by looking at the definition of black swan itself. Taleb developed the concept of black swans largely based on his experiences with VaR models. These models do not accurately take into account fat-tailed events (i.e., low probability, high consequence events). However, homeland security risk analyses are typically focused exclusively on fat-tailed events. When defining scenarios to be analyzed in a homeland security risk analysis, we typically only include these low probability, high consequence events. There is some work now to begin looking at how we can combine high probability, low consequence events (e.g., illegal immigration) into the same risk analyses as low probability, high consequence events, but typically these scenarios are analyzed seperately. In a way, we are already addressing the concerns of Taleb in our risk analyses.
Monday, January 12, 2009
What is Homeland Security Risk Management?
DHS's mission set is vast. It ranges from interdicting illegal immigrants, countering drug smuggling, marine safety, infrastructure protection, cyber security, disaster recovery, airport security, protecting the President, etc.
The broadness of this mission poses some interesting challenges for homeland security risk analysis. Risk analysis is performed in a variety of ways within the Department.... ranging from the tactical-level, where DHS agencies like the Office of Infrastructure Protection perform vulnerability assessments at critical infrastructure, to the strategic-level, where DHS headquarters tries to identify the most important homeland security risks facing the Nation in order to help set planning and budgeting priorities.
Things get especially interesting at the strategic-level. Through its annual budgeting process, DHS must look at its various mission sets and identify which ones should receive more or less funding. Homeland security funds should flow to those programs that have the greatest potential for total risk reduction. Theoretically, a strategic-level risk assessment should support that budgeting process along with various other inputs.
However, conducting a risk assessment that looks across natural disasters, manmade accidents, terrorism, and criminal activities (e.g., drug smuggling) is not easy. Good actuarial data exists for accidents and natural disasters (although global warming trends is beginning to make this data less reliable). We cannot depend on actuarial or historical data for terrorism issues, since terrorists are a dynamic and adaptive adversary. The best we can typically do with terrorism risks is calculate their relative likelihoods, as opposed to their absolute frequencies. But then , how do we compare terrorism and natural hazards risks if we can't put them on the scales?
One approach to comparing natural hazards and terrorism risks is doing "what-if" analyses. The analyst can prepare materials for the decision-maker with several "what-if" scenarios. He/she could prepare graphics that show how terrorism and natural hazards risks compare if the decision-maker believes that there will be a terrorist attack in the next 2 years vs 5 years vs 10 years vs 20 years.
The broadness of this mission poses some interesting challenges for homeland security risk analysis. Risk analysis is performed in a variety of ways within the Department.... ranging from the tactical-level, where DHS agencies like the Office of Infrastructure Protection perform vulnerability assessments at critical infrastructure, to the strategic-level, where DHS headquarters tries to identify the most important homeland security risks facing the Nation in order to help set planning and budgeting priorities.
Things get especially interesting at the strategic-level. Through its annual budgeting process, DHS must look at its various mission sets and identify which ones should receive more or less funding. Homeland security funds should flow to those programs that have the greatest potential for total risk reduction. Theoretically, a strategic-level risk assessment should support that budgeting process along with various other inputs.
However, conducting a risk assessment that looks across natural disasters, manmade accidents, terrorism, and criminal activities (e.g., drug smuggling) is not easy. Good actuarial data exists for accidents and natural disasters (although global warming trends is beginning to make this data less reliable). We cannot depend on actuarial or historical data for terrorism issues, since terrorists are a dynamic and adaptive adversary. The best we can typically do with terrorism risks is calculate their relative likelihoods, as opposed to their absolute frequencies. But then , how do we compare terrorism and natural hazards risks if we can't put them on the scales?
One approach to comparing natural hazards and terrorism risks is doing "what-if" analyses. The analyst can prepare materials for the decision-maker with several "what-if" scenarios. He/she could prepare graphics that show how terrorism and natural hazards risks compare if the decision-maker believes that there will be a terrorist attack in the next 2 years vs 5 years vs 10 years vs 20 years.
Things I Plan to Write About
Here are a few topics that plan on blogging about over the coming weeks/months. I hope to post at least once a week.
- The Homeland Security Risk Equation
- Why Homeland Security Risk Management is Different than Other Types of Risk Management
- Black Swans and Homeland Security Risk Management
- Why Some at DHS Don't Like to Do Risk Analysis
- The Complexities Associated with All-Hazards Risk Assessments
- Measuring Risk Reduction and Performance Management
Introductions
Hello -
I am writing this blog for several reasons.
I am writing this blog for several reasons.
- To clarify and refine some of my thinking by putting pen to paper.
- To educate folks unfamiliar with the topic of homeland security risk management (it is still a developing field and not a ton has been written about it).
- To (hopefully) engage with other folks interested in this topic via the comments section of this blog.
Subscribe to:
Posts (Atom)
Posts
