Protective Clothing Selection
By Gary McEvoy

Step Two: Conduct A Field  (Part Two of Three )
In the first article, we discussed conducting risk assessment as the first step in the protective clothing selection process. Information was provided for how departments should conduct a hazard and risk assessment. This article discusses field testing of prospective personal protective equipment (PPE) – the next step in the selection process.

The overall selection process becomes important when departments begin a new budget cycle and grants flow in. This is when many departments decide to purchase new PPE, including bunker gear, much of it in the next several months.

If your department is in the market for new gear, a properly conducted field test is a critical step to ensuring that you find the PPE which best fits your needs.

The first step in designing an effective field test is to assess your department’s needs. This is typically done by conducting the hazard and risk assessment, as described in the first article. Additionally, your department should outline its bunker gear requirements making sure to include the desired protective properties, features and configurations.

Hazard And Risk Assessment
Determining your department’s needs as part of the hazard and risk assessment is usually conducted by the department’s safety officer with input and recommendations from a PPE committee.
The firefighting and rescue tactics performed and the day-to-day duties of the firefighters wearing the gear are critical factors to consider when assessing requirements for the most effective ensembles.

The frequency of ensemble use should also factor into the decision making process. The average number of runs and the frequency of use of PPE in both training and non-training activities will play a role in developing requirements for your department’s protective ensemble.

When conducting field tests of turnout gear, carefully document the conditions under which they were tested and who was wearing what when the observations were noted.             (Total Fire Group Photo)

Outer Shell Selection
As an example, there are a number of different outer shells to choose from when developing a bunker gear spec. Departments that experience a high volume of activity may choose an outer shell with high abrasion characteristics that extends the life of the bunker gear.

Past experience should be considered. This can run the gambit from first-hand experience on the performance of gear previously worn, to feedback on service and repair issues experienced by your department or a neighboring/mutual aid department.

Pay attention to how long it takes to get your gear after it’s ordered and whether you receive what you ordered and when you needed it. All of these factors should be considered when developing your requirements and deciding which products you want to include in your field test.

Determining Needs
Incident operations will have to be considered as well. Determine for what types of calls you will be wearing the gear, and factor in whether your department uses bunker gear for emergency medical service (EMS) calls and extrications, or whether other gear is worn. This data is critical to designing the requirements of the PPE gear you will be testing.

Without a doubt, geographic location and climate has a huge influence on the type of gear selected. Obviously, gear designed for departments in North Dakota will have different requirements from gear designed for departments in southern California.

The gear weight, thermal protection and breathability are all factors that need to be considered. Consider whether firefighters in your department are more apt to experience climate-related heat stress, or is it more important to protect them from an extremely cold environment.

All manufacturers offer a variety of PPE materials and these materials have differing protective properties. Your risk assessment should provide you with information to determine which combination of materials best fits your department’s needs.

PPE manufacturers are happy to provide departments with information to help develop specifications and are eager to educate departments on what’s new in the marketplace related to features, designs and composites. Most PPE manufacturers are well versed in the new NFPA 1971, 2007 edition that is another requirement to consider.

Remember, the standard is for minimum requirements for structural PPE. Your department should always look for products that exceed the requirements for the utmost protection.

Examine All The Factors
Examining all of these factors, and understanding the tradeoffs and advantages of each element in the initial design phase, is critically important. You must ensure your requirements represent a complete and detailed description of what you are looking for before commencing the field test.

This will allow you to clearly define and communicate your departmental needs and help you determine which PPE manufacturer is best meeting those needs.

It is our intent to do a follow up article that will provide more details on how to conduct a detailed risk assessment. The information outlined above should get you started.

Structuring A Field Test
There are a number of different ways to structure a field test. Some departments prefer to put field test gear on a busy company and let them test the gear in their normal day-to-day activities.

Other departments conduct field tests at training facilities where each field test participant is run through a series of training simulations to determine the gear’s performance levels and the participant’s preferences.

Still other departments use a combination of both the methods. Regardless of the method you decide to use, there are a number of key considerations that you need to include when developing the field test for your department.

First, before the field test is conducted, invite manufacturers to make presentations on the benefits of their prospective products. These invitations also provide an opportunity to lay ground rules with the manufacturers for how they should interact with the department members to keep the selection process above board and transparent.

Next, make sure all the vendor’s gear is included in the evaluation. In order to get the most objective and fair evaluations of gear being tested, it is crucial that all field test participants wear and test each product model being evaluated from each manufacturer.

 

It is also recommended that departments cycle through each model a minimum of three times. This allows a better chance for each participant to experience the gear in various conditions and to validate their initial findings or opinions of each product being tested.

At the end of the field test, data must be compiled and analyzed. Having a quantifiable scoring system in place throughout the testing process with make that job easier. Potential rating categories are provided in Table 1, which accompanies this article.

It is also possible to make physical measurements with the appropriate tools and expertise. For example, tape, rulers, and photography can be used to measure and document rise and reach of members using various sets of gear to see how the gear affects movement.

Numeric Rating Scale
Furthermore, use of a numeric rating scale is recommended for each facet of the gear you are testing, as opposed to a subjective narrative. It is important that when you develop your field test evaluation forms you define the activities you are scoring and what is being measured for each.

Be as specific as possible in what you are asking the participants to measure – and then provide a scale from which the participant can select a score that best matches their experience with the gear being tested.

At the end of each section or component being tested, you may want to provide an opportunity for additional comments that can provide detail on exactly what each participant experienced. Some manufacturers will provide field test evaluation forms with their samples. Most of these forms can be provided electronically and will supply you with either a ready to use tool or a starting point for developing a form that fits your needs.

Define The Timeframes
If you want your field test to run smoothly, you will need to define all of the timeframes and develop a schedule that includes all of the activities and key milestones associated with your field test.

You should determine in advance how long the test will last and when the participants will cycle through each product. It’s also important to determine exactly when the evaluation forms will be filled out and when the gear will be collected. Your department will need to determine when the safety officer and/or PPE committee will analyze the results and present its findings, when they will fill out evaluation forms, and present their recommendations.

All of this information should then be communicated to everyone included in the test. It is important that the field test be transparent within the department so it is clear that an objective process has been used.

The primary reason for conducting a field test is to generate a recommendation for a product that best fits your department’s needs. This recommendation will be delivered in a final report, which should be used to develop purchase specifications.

When putting together your field test parameters, you should determine in advance who will be responsible for developing the final report and any interim reports associated with the testing and what will be included in the reporting at each stage.

It is also advisable to determine who will have access to these reports along the way, as well as access to the final report. A well-documented field test that results in a recommendation based on quantifiable results can be a powerful tool in providing justification for the purchase of gear that will be the best solution for your department.

According to NFPA 1851, Standard on Selection, Care and Maintenance of Structural Fire Fighting Protective Ensembles, 2001 edition; “Test participants should be selected based on a cross section of personnel, willingness to participate, objectivity and level of operational activity.”

Use Experience Members
This means you should select participants for your field test and include members who are experienced and know what to look for in proper functioning bunker gear. These participants should also have an open mind about all of the products being tested. You obviously don’t want to choose a participant for an objective field test who has already decided that one product is really the only clear choice before beginning the evaluation.

You also want to make sure the selected participants are involved in all areas of your department’s operational activity and are willing and able to commit time to the testing process and completing the field test evaluation forms in a timely manner.

Plan Kick-Off Meeting
The initial step in the implementation of your field test should be a kick-off meeting. In this meeting, the safety officer will provide participants with details of the field test to ensure they know what the objective is and how the test will be conducted.

The risk assessment, field test schedule, participant requirements and an explanation of products being tested are all subjects that should be covered in this meeting.

Prior to this meeting, you should have the product manufacturers that will be included in the test measure the participants and then come in to fit the gear and instruct participants on how to properly wear the gear.

This is the time to let the manufacturers know what will be expected of them and to provide rules preventing undue influence in their effects on the field test. In other words, “the products should speak for themselves.”

It is recommended that the participants only wear and retain one set of the test gear at a time to minimize confusion and the chances that all gear is equally tested. You can distribute the gear for the initial round of evaluations at the kick-off meeting.

In accordance with your field test schedule, at preset intervals, the Safety Officer should provide participants with evaluation forms and have them completed and returned where they can be filed for future analysis. This can be done in person or electronically.

The safety officer should not provide the next gear in the rotation until the evaluation forms are completed for the current gear being tested. Evaluation forms should be filled out throughout the test – it is recommended by NFPA 1851 that three evaluation forms per ensemble or product be completed by each participant.

The safety officer should also establish a process for the reporting of events that fall outside the normal reporting procedures. Examples of these events may include the need to repair or replace field test gear, any service related issues associated with the field test gear or any critical information that comes to light as a result of the field test that may not be included on the field test evaluation forms. These events should be documented by the Safety officer and included in the data used to compile the final report.

The participants for your field test should remain consistent throughout the test. If for some reason one participant needs to drop out of the test, it is better not to replace that participant and to discard any evaluation forms submitted by that participant unless you have a complete cycle of evaluations from that participant for all gear being tested.

If, based on a limited number of participants (less than four in total) you need to substitute field test participants, do so only at the beginning of a new cycle.

The safety officer should collect and retain all field test gear once the field test is completed so that the selection committee can examine all samples before developing a formal recommendation.

Formal Recommendation
Once the official testing in completed, it’s time to compile the results and develop a formal recommendation.

Before writing your formal recommendation, be sure to take time to look at all the information and results your field test has generated. If your field test evaluation forms provide quantifiable scoring data, then it will be easier for you to calculate the results for each category you tested, but you should not limit your report to just the information generated by the evaluation forms.

Safety Officer Inspection
The Safety Officer and/or PPE Committee should also perform a thorough inspection of all sample gear used in the field test. Any finding associated with wear, component failure or workmanship should be included as part of the final review and any resulting recommendations.

In addition, the safety officer should provide any information pertaining to the field test as it relates to service, delivery, repair or accuracy to specification for inclusion in the final recommendation.

The most effective recommendations begin with an overview of the requirements that were developed based on the initial risk assessment. This sets the stage by letting report readers know exactly what the gear committee was looking for and why.

This section is then followed by results of the field test by category tested. In most cases, departments tabulate all scores and then provide an average product score for each question on the evaluation form.

Any participant comments are then listed by manufacturer at the end of each section of the report. The overall organization of the report will be based on how the field test was conducted and how the evaluation forms were designed.

The final section of the report should be a formal recommendation of a specific product, which includes the rationale of why that specific product was selected.

Develop A Formal Bid
This recommendation should form the basis for the development of a formal bid for the product that you tested. If your bid specification does not align with the results of your field test then you are probably not going to get the product that best meets your department’s needs.

A future article will describe the details of designing a formal bid specification around the results of your field test.

In conclusion, the use of a field test is an integral and systematic part of the process for selecting PPE. Field testing offers the best way for your department to assess the different brands available and to determine how well that gear will live up to the intended and expected performance.

The proper design of a field test that is quantitative, balanced, and transparent allows your department to provide documentation for obtaining the gear that best meets your department’s needs.

Editor’s Note: Gary McEvoy is vice president of marketing for Total Fire Group/Morning Pride, a position he’s held since 2002. In that role, he directs all of the Total Fire Group’s regional directors and inside marketing associates, trade show coordination, field administration, bidding and contract administration.