[CPEO-MEF] Analysis of proposed Munitions Response Prioritization Protocol

[Lenny Siegel <lsiegel@cpeo.org>]

The Defense Department's proposed "Munitions Response Site
Prioritization Protocol," which appeared in the August 22, 2003 Federal
Register (pp. 50900 ff.), is complex but comprehensible. Required by
legislation originally introduced by Rep. Earl Blumenauer (D-Oregon),
the Protocol will be a significant step forward in establishing a
systematic national munitions response program. I provide a brief
summary and a few comments below, but the best way to understand the
proposal is to read the Federal Register notice, which may be downloaded from
http://a257.g.akamaitech.net/7/257/2422/14mar20010800/edocket.access.gpo.gov/2003/pdf/03-21013.pdf. 
The Department of Defense is accepting comments through November 20, 2003.

The Protocol uses a quantitative rating scheme to classify, into one of
eight categories, all munitions response sites. This includes former
ranges and other sites known or suspected to contain unexploded
ordnance, discarded military munitions, chemical weapons materiel, or
munitions constituents. It does not include operational ranges, but it
might include munitions disposal sites located within the boundaries of
operational ranges. The munitions response sites may be on active
installations - the military has identified over 500 thus far in its
official inventory - or Formerly Used Defense Sites (FUDS) or other
closed facilities.

It's important to recognize that the Prioritization Protocol does not
evaluate risk for the purpose of selecting a remedy. It does not purport
to establish a decision-making process for determining how to respond to
known or suspected munitions contamination. It is designed to provide an
initial basis for sequencing decisions, but drawing upon the consensus
recommendations of the 1996 report of the Federal Facilities
Environmental Restoration Dialogue Committee (FFERDC), the Department
recognizes that other factors may influence the timing of response activities.

The Protocol, once adopted, will direct Defense components to divide all
munitions response areas - ranges, etc. - into munitions response sites
to be rated under the proposed system. Each site requiring a munitions
response will be assigned a priority level, ranging from 1 (highest) to
8 (lowest). That level will be derived from the highest priority (lowest
number) that the site receives after evaluation using three distinct 
evaluation modules: Explosive Hazard Evaluation (EHE), Chemical Warfare
Materiel (CWM) Hazard Evaluation (CHE), and Relative Risk Site
Evaluation (RRSE). Chemical Warfare Materiel refers to lethal chemical
agent. The Relative Risk Site Evaluation, already used by the Defense
Department to prioritize non-munitions cleanup sites, will be used to
rank the hazards posed by munitions constituents. 

Both the EHE and CHE classify sites into seven categories, rated A
through G. The RRSE, as already in practiced, divides sites into High,
Medium and Low. The Protocol provides a table that converts those
rankings to the eight priority levels. Rather than combine Evaluation
Model rankings with a mathematical formula, the Protocol directs that
the combined level be the highest of the three ratings. Only sites with
the highest CHE scores may qualify for priority level 1. All sites with
chemical agent hazards will rank level seven or higher priority (a lower number).

The Explosive Hazard Evaluation module is designed to score, on a scale
of 100, the relative potential explosive hazards at a site. The first
factor, Explosive Hazard, combines up to 30 points from two date
elements, Munitions Type and Source of Hazard (type of range, treatment
area, or other facility). The second factor, Accessibility, combines up
to 40 points from three data elements: Location, Ease of Access, and
Status of Property. The Location element considers if ordnance is found
on the surface, whether its presence is confirmed or suspected, and if
geological processes are likely to expose buried ordnance in the future.
Status of Property simply describes whether the site is under Defense
Department control. The Receptor factor is designed to generate a score
of up to 20. Its four data elements include Population Density,
Population (number of structures) Near Hazard, Types of
Activities/Structures (within two miles), and Ecological and/or Cultural Resources.

The Chemical Warfare Materiel Hazard Evaluation also is on a 100-point
scale, divided into the same three factors: Hazard, Accessibility, and
Receptors. The CWM Hazard Configuration data element scores up to 30
points, depending on the condition or the munition and whether it
contains explosives as well as chemical agent. The Hazard Source data
element allows up to 10 points based upon the nature of the facility.
The Accessibility factor, like the EHE module, assigns up to 40 points
to Location, Ease of Access, and Status of Property. The Receptor
factor, also like the EHE module, allots up to 20 points to Population
Density, Population Near Hazard, Types of Activities/Structures, and
Ecological and/or Cultural Resources.

Finally, the Relative Risk Site Evaluation Module qualitatively combines
a Contaminant Hazard Factor, Migration Pathway Factor, and Receptor
Factor. Sites are labeled High, Medium, or Low risk. The RRSE has been
applied for several years to Defense Environmental Restoration Program
sites that have been found to require some form of cleanup.


***

Upon my initial review of the proposed Protocol, I find it sensible and
straightforward. It is inclusive, in that it appears to include all
factors one would expect in a munitions response priority-setting
system. It is replicable, which means that the ranking of a site, at
different times or by different people, is likely to come up with the
same result. And it's transparent. A typical public stakeholder should
be able to understand not only the protocol but the ranking results for
any site. Furthermore, the Defense Department tested the protocol at
over 70 sites to ensure that there is a useful spread - that is, that
most of the sites don't end up in the same category. (There was a small
problem with the RRSE when first implemented. Too many sites ended up in
the high-risk category.)

Quantitative prioritization scheme make me nervous, because there is
generally no underlying objective basis for the numbers assigned.
Rather, the scores are the best subjective efforts of a collection of
experts. For example, there is no inherent reason why three points, as
opposed to another number, separates residential uses from industrial
properties. (It's 5 vs. 2.) In another data element, there's no
scientific basis for counting the population within two miles, as
opposed to one mile or five miles.

The developers of the Protocol have attempted to overcome that
difficulty through their testing program. That is, they applied the
scoring system to more than 70 sites, and they considered whether the
relative scores made sense. Perhaps they could further "ground-truth"
the test results by asking external stakeholders, particularly state
regulators familiar with many of these sites, to review the scores. The
Defense Department consulted extensively with state representatives and
others while constructing the protocol. Asking them to validate test
results would be a logical next step.

Perhaps more important, the Protocol calls for use of the "risk-plus
other factors" approach suggested by FFERDC. Once each site has a
relative priority level, other factors may still influence the timing of
the response. These include:
* Concerns expressed by stakeholders.
* Cultural and social factors.
* Economic factors
* The reasonably anticipated future land use
* Implementation and execution considerations 
* The availability of technology
* Implementing standing commitments 
* Tribal trust land requirements
* Ecological impacts

The preamble to the proposed Protocol provides an example of how the
consideration of another factor may help overcome the limitations of the
quantitative scoring system. A number of commenters suggested that daily
visitor counts to national recreational areas be considered as a
possible substitute to population in evaluation receptors. The Defense
Department rejected this idea as too difficult to implement. Instead, it
suggested that the risk-plus approach would allow the presence of
transient populations to be incorporated into sequencing decisions.

The risk-plus approach should also help overcome the limitations of the
Protocol at properties proposed for reuse. While the proposed Protocol
considers ongoing construction in the EHE and CHE Location data
elements, there is no mechanism for moving up the priority of a property
where the owner or prospective owner plans redevelopment. That's
probably as it should be. Deciding to conduct extensive ordnance
clearance at a site simply because there is a redevelopment proposal is
ultimately a political issue that cannot be solved with a scoring system.

I believe the Protocol could strengthen its discussion of uncertainty,
in particular the scoring of sites where the precise nature and extent
of ordnance is unknown. At ranges and disposal sites, cleanup teams
don't know exactly where munitions remain until their work is done. And
since current disposal practices are based on assuming the worst - for
safety reasons - the contents of weapons are sometimes never known. That
is, ordnance specialists often assume that what might be inert practice
bombs contain high explosive fillers, and (unless a sophisticated test
is conducted) that projectiles designed to contain liquids still contain
chemical agent. It may make sense to assume the worst when scoring
sites, too, but I couldn't find any language that clearly addresses that
problem within the proposed Protocol

The only disturbing information in the Federal Register notice of the
Protocol was the Defense Department's timetable for implementation. The
Department proposed a milestone of May 31, 2007 for completing at least
one evaluation module at each site not already evaluated under existing
priority systems - the RRSE and the Army Corps of Engineers' Risk
Assessments Code - and a May 31, 2012 goal for completing all evaluation
modules. 

That's much too slow. Of course, the problem isn't so much with policy,
but with the shortage of funds for munitions response. If Congress wants
the entire inventory prioritized, it will have to allocate sufficient resources.

In closing, developing this proposed rule was a major undertaking. It's
long and detailed. Fortunately, the comment period runs until November
20. I invite people who worked on the proposal to correct any
misinterpretations I may have made or to highlight important features
that I missed. And I invite all readers of the CPEO Military
Environmental Forum to submit questions or offer comments for general discussion.

Lenny Siegel
September 4, 2003

-- 


Lenny Siegel
Director, Center for Public Environmental Oversight
c/o PSC, 278-A Hope St., Mountain View, CA 94041
Voice: 650/961-8918 or 650/969-1545
Fax: 650/961-8918
<lsiegel@cpeo.org>
http://www.cpeo.org

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CPEO: A DECADE OF SUCCESS.  Your generous support will ensure that our 
important work on military and environmental issues will continue.  
Please consider one of our donation options.  Thank you.
http://www.groundspring.org/donate/index.cfm?ID=2086-0|721-0