Q&As: STAR Program
1. General STAR Questions
2. Station Types
2.7 I own a Smog Check station in a Change-of-Ownership Area. Can I still become a STAR certified station even though the majority of vehicles my station inspects are not in a biennial inspection (Enhanced or Basic) area?
3. STAR Certification
4. STAR Scores
5. Short-Term Performance Measures
MAX READINESS MONITORS
5.10 When I repair a failing vehicle, I make sure all of the OBD II monitors have run prior to certifying it. Other stations, however, may follow different procedures. Will my STAR score be affected for certifying repaired vehicles with unset readiness monitors?
INCORRECT GEAR SELECTION
6. Follow-up Pass Rate (FPR)
6.2 If a STAR certified station has an inspector entered into their analyzer whose FPR score drops too low (less than 0.10), will the station automatically be ineligible to certify directed and gross polluting vehicles?
6.11 If the FPR evaluates how vehicles previously certified by stations and inspectors are performing in the current inspection cycle, how will new stations and inspectors be evaluated under this performance measure?
1. General STAR Question1.1 What is the STAR Program?
The STAR Program is a certification program for both Test-Only and Test-and-Repair Smog Check stations that wish to certify directed and gross-polluting vehicles. STAR Test-and-Repair stations also must provide repair assistance services under BAR's Consumer Assistance Program.1.2 When did the STAR Program begin?
The STAR Program began January 1, 2013.1.3 How will consumers know which stations are STAR certified?
The Smog Check station locator tool on BAR's website is the most convenient and up-to-date way for consumers to identify STAR stations in their area that can inspect their vehicles.1.4 How are the standards for the STAR performance measures generated?
Standards for the STAR performance measures are set by evaluating stations and inspectors against other stations and inspectors throughout the Smog Check Program. For an explanation of how each STAR performance measure is evaluated, visit the STAR Report Card web page.1.5 Can inspectors be STAR certified?
No, only stations can be STAR certified. However, an inspector’s long term performance measure, the FPR, is one of the factors used to determine station eligibility for STAR (see Questions 4-6 for details).1.6 Are there any plans to change the percentage of "directed vehicles
There are no plans to change the percentage of "directed vehicles."
2. Station Types2.1 What station types are in the Smog Check Program?
Smog Check stations can be licensed as either a Test-Only, Test-and-Repair, or Repair-Only station. Both Test-Only and Test-and-Repair stations may apply for STAR certification. As a result, there are four possible station types allowed to inspect vehicles:
- STAR Test-Only: cannot repair vehicles, but can inspect and certify all vehicles including directed and gross-polluting vehicles;
- STAR Test-and-Repair: can repair vehicles, can certify directed vehicles and gross-polluting vehicles, and must provide repair assistance services under BAR's Consumer Assistance Program;
- Test-Only (non-STAR certified): can inspect and certify vehicles that are not directed to STAR stations and/or vehicles that are not identified as gross-polluting during their initial inspection. Vehicles inspected apart from their biennial inspection requirements (e.g., change-of-ownership) are not directed to STAR stations.
- Test-and-Repair (non-STAR certified): can inspect and certify vehicles that are not directed to STAR stations and/or vehicles that are not identified as gross-polluting during their initial inspection. Vehicles inspected apart from their biennial inspection requirements (e.g., change-of-ownership) are not directed to STAR stations. Non-STAR Test-and-Repair stations can repair both directed and non-directed vehicles.
- Repair-Only: are licensed only to perform repairs, and not Smog Check inspections. As such, Repair-Only stations are not eligible for the STAR Program.
No, only STAR certified stations may inspect directed and gross-polluting vehicles.2.3 Can I own both a Test-Only station and a Test-and-Repair station, regardless of whether or not the stations are STAR certified?
Yes, but with some restrictions. If the two stations have a common financial interest and are located adjacent to each other, or in the same business park, strip mall, or industrial complex, one station cannot be licensed as a Test-Only station and the other licensed as a Test-and-Repair station. Instead, they both must be either licensed as a Test-and-Repair station or both licensed as a Test-Only station.2.4 Can Test-Only stations apply for STAR certification?
Yes, there are no repair-based performance measures in the STAR Program that would prevent Test-Only stations from applying for STAR certification.2.5 Can a Test-Only station apply for certification as a STAR Test-and-Repair station?
Yes, since there are no repair-based performance measures in the STAR Program. A Test-Only station can change its license to a Test-and-Repair station and apply for the STAR Program.2.6 Can a STAR Test-Only station change to a STAR Test-and-Repair station?
The new license and certification will be issued only after a BAR representative performs an onsite inspection of the station and the required Consumer Assistance Program Standard Agreement is signed and returned to BAR.2.7 I own a Smog Check station in a Change-of-Ownership Area. Can I still become a STAR certified station even though the majority of vehicles my station inspects are not in a biennial inspection (Enhanced or Basic) area?
Yes. Stations located in a Change-of-Ownership Area still will have the opportunity to participate in the STAR Program. However, one of the rewards of the Program - the ability to certify directed and gross-polluting vehicles - may not have as much value to a station located outside of an Enhanced Area, or even a Basic Area. Scores for both stations and inspectors from Change-of-Ownership Areas will be posted on the STAR Report Card web page along with scores for stations and inspectors from the Enhanced and Basic Areas of the state.
3. STAR Certification3.1 How long does a station have to be in business before it can become STAR certified?
Stations may be eligible for STAR certification once they have a passing score on the short-term measures from the previous calendar quarter. View your score by visiting the STAR Report Card web page on BAR's website.3.2 Can a station's STAR certification be transferred to a prospective buyer or new owner of the station?
No. A station's STAR certification is not transferable to a prospective buyer or new owner. Upon change of ownership, the new owner must apply for a new automotive repair dealer registration and Smog Check license and may apply for STAR certification when eligibility requirements are met.3.3 Will my station's STAR certification be affected if I change the station's business structure?
Yes. Business structure changes such as changing to a sole proprietorship, partnership, corporation or vice versa, which result in the issuance of a new license, will require the newly licensed station to meet the STAR performance measures to qualify for STAR certification using data from the new license and data from the previous license. A business structure change does not include situations where a corporation changes corporate officers.3.4 Will my station's STAR certification be affected if I relocate to a new address?
No. Changing a business address will not affect a station's STAR certification. However, your station will experience some downtime until an onsite inspection at the new location is performed by BAR.3.5 Will my station's STAR certification be affected if I fail to renew my automotive repair dealer registration and/or Smog Check station license on time?
Failure to renew your station licenses, including both your automotive repair dealer (ARD) registration and Smog Check station license, prior to the expiration date will cause the station to be delinquent. This will, in turn, lock the station analyzer out from performing smog inspections until both the ARD registration and Smog Check station license are current.3.6 My Smog Check station offers a "free retest" to consumers who fail their initial Smog Check at my station. Will this policy affect my STAR certification, and if so, how?
Stations offering "free retests" provide two inspections for the price of one whenever a vehicle fails its initial inspection. This can cut into station profits, especially at busy stations where shop owners must forgo additional work to perform the retests. A station's eligibility for the STAR Program could be jeopardized in situations where stations and inspectors perform incomplete or inaccurate Smog Check inspections to avoid having to provide "free retests."3.7 My station's STAR certification was invalidated. When is my station eligible to re-apply for the STAR Program?
Stations that have their STAR certification invalidated by BAR are not eligible to apply for the STAR Program for at least six months. A station or any of its inspectors also cannot have received a citation within the last year or have been the subject of a BAR administrative action with the last three years. Both are based on the effective date of the citation or administrative action, not the date of issuance.
4. STAR Scores4.1 What source of data is used to generate the STAR performance scores?
All STAR performance scores are calculated using data collected by the Smog Check inspection equipment during a vehicle inspection.4.2 How can stations and inspectors find their STAR Program scores?
Visit the STAR Report Card web page to check STAR scores.4.3 How often are STAR scores updated?
Short-term measures, which include the Test Deviations, Similar Vehicle Failure Rate (SVFR), and Improper Gear Selection during the ASM test, are updated every month. However, pass/fail decisions are made only on a calendar quarter basis. The Follow-up Pass Rate (FPR), which is a long-term performance measure, is calculated and updated twice a year, in January and July.4.4 Can station owners/managers use BAR's STAR Report Card web page to determine an inspector's STAR score?
STAR scores are provided for all stations and inspectors licensed to perform Smog Check inspections. Stations looking to enter an inspector into their station analyzer(s) may choose to check an inspector's FPR score since each inspector's FPR score will directly affect the station's STAR certification. Station owners can check an inspector's performance on the other STAR measures that the station must meet to make sure that the inspector is less prone to adversely affect a station's STAR scores.4.5 Can consumers use BAR's STAR Report Card Web page to find stations and inspectors that are more likely to perform a low quality inspection?
BAR is aware that some consumers may try to use the STAR Report Card web page to identify lower-performing Smog Check stations that are more likely to perform an improper Smog Check inspection. To reduce the likelihood of this happening, the STAR Report Card web page has been designed to require that the user know the station or inspector license number in order to conduct a search. While this will not prevent unintended use of the STAR Report Card web page, it will make it less convenient, and hopefully, less common.
5. Short-Term Performance Measures
GENERAL5.1 Where can I learn more about STAR Program performance measures? 5.2 Are STAR Test-and-Repair stations evaluated for repair effectiveness?
Repair effectiveness is not evaluated as part of the STAR Program. AB 2289 (Eng, Chapter 258, Statutes of 2010) specifically precludes BAR from using different performance measures for both Test-Only and Test-and-Repair stations in the STAR Program. Since Test-Only stations do not perform repairs, we cannot use repair effectiveness as a performance measure for any station type in the STAR Program.5.3 Will my station's STAR certification be affected if an inspector at my station receives a citation?
It depends on the citation. Applicable citations are detailed in §3392.5.1(a)(1) of the California Code of Regulations. Note that among the applicable citations, a single citation with an order of abatement issued to an inspector typically would not affect a station's STAR certification. Testing a 2000 and newer model-year gasoline-powered vehicle, or 1998 and newer model-year diesel vehicle on the BAR-97 Emission Inspection System (EIS) instead of the required OBD Inspection System (OIS) usually results in a citation with an order of abatement. However, citations for other reasons or multiple citations for failing to test on the OIS can be cause for invalidation of a station's STAR certification.5.4 Why is it important to use updated Smog Check reference materials?
As required by California Code of Regulations sections 3340.16 (a) and 3340.16.5 (a), stations and inspectors should always use up-to-date reference materials when performing a Smog Check inspection. This includes the Smog Check Manual and emission control system application guides. Failure to use up-to-date material can result in improper inspection procedures that may lead to stations failing to meet the STAR performance standards.5.5 Can one isolated mistake on a Smog Check inspection force me out of the STAR Program?
It depends on the nature of the mistake. A mistake that leads to a citation or an administrative action is grounds for and can lead to invalidation of a station's STAR certification. In contrast, it generally takes several repeated inspection errors before a station fails to meet any of the STAR performance measures that are based on Smog Check inspection data. Note that typically a single citation with an order of abatement, such as for testing a 2000 and newer model-year gasoline-powered vehicle, or 1998 and newer model-year diesel vehicle on the BAR-97 Emission Inspection System (EIS) instead of the required OBD Inspection System (OIS), would not result in the invalidation of a station's STAR certification. However, citations for other reasons or multiple citations for failing to test on the OIS can be cause for invalidation of a station's STAR certification.5.6 Will my Similar Vehicle Failure Rate (SVFR) be affected if I replace a failing fuel cap in the middle of an inspection and then pass the vehicle?
Smog Check inspectors have the option to replace leaking fuel caps mid-inspection in order to avoid having to fail the vehicle when the sole reason for the Smog Check failure is the fuel cap. When this occurs, and the inspector indicates that the fuel cap was replaced during the test, the SVFR performance measure will treat the inspection as though the fuel cap failed the leak-down portion of the inspection. Thus, while the official result for the inspection was a pass, replacing the cap mid-inspection will not affect a station's SVFR.5.7 What should I do if I think an incident for one of the short-term measures is incorrect?
The Report Card web page provides information related to vehicle testability on certain elements of the Smog Check inspection based upon results from the industry for similar vehicles. However, BAR is providing stations and inspectors an opportunity to challenge an incident in situations where the STAR Report Card web page indicates that a station or inspector failed to perform a specific element of the Smog Check inspection and the station or inspector can prove that the test was not appropriate.
For example, if an incident is shown for your station or an inspector not having performed a timing test on a vehicle when the vast majority of other stations or inspectors did perform a test on similar vehicles, and the vehicle did not have adjustable timing, then the specific incident for that vehicle can be challenged.
For station owners who feel that an incident is incorrect and is keeping the station from qualifying for the STAR Program, the station owner should submit an application for STAR certification. If the application is denied by BAR's Licensing Unit, specific instructions will be provided detailing the process for having a re-evaluation of the application denial. Note that station incidents will be considered if they are keeping a station from passing the STAR performance measures.
For inspectors who wish to challenge an incident, send an email to BAR's Help Desk at firstname.lastname@example.org. Inspector incidents should only be challenged if they cause the result for a specific performance measure to be highlighted in red on the inspector's report card.
Challenges to specific incidences, either by the inspector through BAR's Help Desk or by the station while requesting a re-evaluation of an application denial, will only be considered if the challenge is specific and supported with appropriate documentation. For example:
"The STAR Performance Report Card indicates that I should have performed a timing test on a [model-year, make, model, certificate number issued] vehicle. However, as indicated in [provide a detailed reference for your information such as ALLDATA or Motor, including links or page numbers], this vehicle does not have adjustable timing and, for this reason, should not have a timing test."
BAR encourages the industry to submit well-researched information regarding vehicle testability, as this information will be used to refine future analyses under the STAR Program. For this reason, requests to challenge an incident will not be considered if the basis for the challenge has not been researched and supported with specific documentation.5.8 How does the STAR Program account for different socio-economic areas in which stations operate?
The STAR Program does not consider geographically-based factors to identify vehicle owner means or affluence. Instead, the STAR Program uses the vehicles themselves, by comparing vehicles with similar model-year, make, model, engine displacement, mileage, etc. The reasoning for this analysis is that the characteristics of the vehicles, regardless of where they are registered, are better indicators of consumer behavior and one's financial means than is the address at which one resides. For instance, it is highly likely that the average owners of 1991 Saturn Coupes with 150,000 miles will have a lot more in common with each other than people who just happen to live within a mile of each other. In other words, the best information available to help correct for differences in station demographics are the vehicles themselves. For this reason, all of the STAR performance measures take into account the specific vehicles inspected by each station, and compares to similar vehicles inspected statewide.
MAX READINESS MONITORS5.9 Is the Max Readiness Monitor performance measure still in effect since the implementation of the new readiness standards for OBD II inspections?
On May 4, 2015, BAR implemented new readiness monitor standards for OBD II inspections. In coordination with that change, Max Readiness Monitors ceased to be a STAR performance measure. This change is reflected on the STAR report card beginning with the second quarter 2015 scores. Stations will no longer receive a STAR test deviation for a high Max Readiness Monitor rate. Note that for informational purposes, the STAR report card will continue to display the Max Readiness Monitors measure, including the percentage of initial inspections at the maximum readiness limit for each station and inspector, and for similar vehicles statewide.5.10 When I repair a failing vehicle, I make sure all of the OBD II monitors have run prior to certifying it. Other stations, however, may follow different procedures. Will my STAR score be affected for certifying repaired vehicles with unset readiness monitors?
Vehicles with underlying defects that are certified even though the offending OBD II monitor hasn't yet run will tend to fail their initial inspection at a higher rate in the next inspection cycle. This will tend to drive down the FPR scores for stations and inspectors that previously certified these vehicles.5.11 Should I pre-screen vehicles for unset readiness monitors to avoid an impact to my STAR score?
Stations concerned that their STAR scores could be affected by unset readiness monitors, either because they certify a lot of vehicles for third parties or because already published scores show this to be an issue at that station, may choose to prescreen OBD II systems for unset readiness monitors. Stations may have OBD II scanners available for the purpose of pre-screening vehicles for OBD II readiness.
INCORRECT GEAR SELECTION5.12 My BAR-97 emissions inspection system often has difficulty picking up a stable engine RPM signal for vehicles with distributorless ignition systems (DIS). Will this affect my STAR score?
The one STAR performance measure that uses the engine RPM reading is "Incorrect Gear Selection." For that measure, ASM test results accompanied by abnormally high RPM readings are flagged as having been shifted into the incorrect gear during an ASM test mode.
Normally, antenna-based RPM errors will not result in Incorrect Gear Selection incidents. Antenna-based errors typically drive the RPM low, not high, as the RPM reading often drops out completely. Low readings will not cause Incorrect Gear Selection incidents. Furthermore, results for this measure are corrected for the specific vehicles inspected by each station. In other words, results for specific vehicles for which RPM readings are difficult to measure will be compared to results from similar vehicles, thus the performance standards will be appropriately based upon the vehicles inspected by each station.
Still, BAR recommends that stations pull the RPM readings directly from the OBD II system whenever possible. For CAN vehicles, which often have DIS and for which original BAR-97 emissions inspection systems cannot communicate through the OBD II port, stations may want to consider purchasing the BAR-97 update. This will allow inspectors to collect RPM readings for CAN vehicles directly using the OBD II port.5.13 For the Incorrect Gear Selection measure, will a station be penalized if many of the vehicles it inspects are equipped with non-factory sized tires?
No, the Incorrect Gear Selection measure is fairly lenient and allows for some variation in tire size without identifying an inspection as being driven in the incorrect gear. The underlying question is whether or not inspectors can alter the Smog Check procedures to accommodate a vehicle that is equipped with tires so large or small that it cannot be tested within the correct RPM range in the specified gear. Smog Check inspectors are required to follow the procedures outlined in the Smog Check Manual. If the vehicle is tested "as is" please note that it could affect the station's performance on the Incorrect Gear Selection measure, in which case it should be documented on the Vehicle Inspection Report. The station also has the option of referring the vehicle to the Referee for inspection.5.14 Does the type of transmission entered during an ASM inspection affect the Incorrect Gear Selection performance measure?
Yes. If an inspector enters a manual transmission vehicle into the EIS as an automatic transmission, the inspection will likely trigger a gear shift incident. This is due to the fact that the measured RPM for a manual transmission vehicle tested in 2nd gear during the "2525" mode of the ASM test will often fall outside the allowable range for automatic transmission vehicles. This type of error will be apparent when looking at the individual incidents under the Incorrect Gear Selection performance measure because the RPM during the "2525" portion of the ASM test will be roughly two-thirds (2/3) higher than the RPM of the "5015" test mode. The way to correct this problem is to be sure to enter into the EIS the correct transmission type for each vehicle receiving an ASM test. As a general practice, BAR also recommends carefully reviewing all vehicle data entered into the EIS prior to beginning the ASM test mode.5.15 During an ASM test, in which gear should vehicles with automatic transmissions be inspected when the "Drive" position is not clearly identified on the vehicle?
Some newer automatic transmission vehicles, especially those with more than four forward gears, may not have clearly labeled "drive" settings. When the "drive" setting is not clearly identified on a vehicle, automatic transmission vehicles should be driven in the gear selector's default driving position. This means the position where the vehicle automatically shifts through the forward gears during the course of driving.5.16 During an ASM test, in which performance mode should vehicles with user-selectable automatic transmission shift settings be driven?
Some automatic transmission vehicles have different performance modes or settings that control the automatic transmission shift points. These are typically the `standard/economy" mode or the "sport/performance/tow" mode. In the standard/economy setting, shift points will usually occur at lower engine RPM than they will with the sport/performance/tow settings. During an ASM test, vehicles equipped with user-selectable transmission performance settings should be placed in the economy/standard mode resulting in lower engine RPM shift points.5.17 During an ASM test, how should a vehicle equipped with a transmission that allows either manual or automatic shifting be inspected?
If a vehicle is equipped with a transmission that was designed to shift automatically, the vehicle shall be ASM tested in the mode allowing automatic shifting, even if the vehicle has an option for manual shift operation. Vehicles that do not have a fully automatic setting shall be tested as a manual transmission, even if they do not have a manual clutch.
6. Follow-up Pass Rate (FPR)6.1 Can I participate in the STAR Program if I have an inspector entered into my analyzer who has a low FPR score?
If a station has an inspector entered into their analyzer with an FPR score that does not meet STAR standards, the station may have difficulty having their application approved for the STAR Program. Additionally, a STAR certified station may be in jeopardy of having its STAR certification invalidated by BAR for having an inspector entered into their analyzer with an unacceptable FPR score. Refer to the FPR Eligibility Chart for more information.6.2 If a STAR certified station has an inspector entered into their analyzer whose FPR score drops too low (less than 0.10), will the station automatically be ineligible to certify directed and gross polluting vehicles?
Station's are subject to having their STAR certification invalidated any time an inspector entered into their analyzer does not meet the FPR score requirements. The invalidation process will include notification to the station and an opportunity to mitigate the issue or to appeal the proposed decision.6.3 Is a station's eligibility for the STAR Program based on an average of the FPR scores of all inspectors working at the station?
No, scores of each individual inspector entered into the station's analyzer(s) must meet the FPR criteria. Once a station is STAR certified, BAR considers any inspector entered into the station's analyzer(s), even if that inspector has only been entered into the analyzer(s), for one day during the evaluation period.
Consider the following example: a station has five Smog Check inspectors entered into their analyzer(s), four of whom have perfect FPR scores of 1.00, and the fifth inspector has an FPR score of 0.00. This station would be denied STAR certification because one inspector has too low of an FPR score. This station would be eligible to apply for the STAR Program once the low-scoring inspector is removed from the stations analyzer(s).6.4 What are my options if I am both a station owner and inspector with a low FPR score?
A station owner who is also an inspector with a low FPR score may do one of two things. One, the owner may choose not to have their station participate in the STAR Program at this time. Two, enter another inspector into the station's analyzer(s) and remove all inspectors with low FPR score. The owner/inspector with an FPR score below the STAR standards must improve their score to acceptable levels before they can be entered into the station's analyzer(s) to participate in the STAR Program.6.5 How do stations or inspectors improve their FPR score?
FPR scores can be improved by ensuring that accurate Smog Check inspections are performed according to the Smog Check Manual.
Specific behaviors that affect a station's or inspector's FPR score include:
- Clean piping (i.e., passing a vehicle that is out of compliance with the tailpipe emissions standards by introducing a substitute clean exhaust sample through the emissions analyzer)
- Clean plugging (i.e., using a substitute source of OBD II data for a failing vehicle's OBD II self-diagnostic test)
- Shifting vehicles into the incorrect gear during an ASM test
- Over-conditioning vehicles (i.e., racing the engine to get a vehicle's catalytic converter hotter than would happen under normal operating conditions)
- Not identifying visual inspection failures
- Not identifying functional inspection failures (e.g., fuel cap, ignition timing, low-pressure fuel evaporative emissions)
- Entering incorrect vehicle parameters to generate more lenient emission standards or a lighter vehicle weight loading (in order to create less treadmill resistance) during an ASM test
Most stations and inspectors can expect to see their scores improving two years after changing a behavior that negatively impacted their score. Note, however, that when calculating the FPR, BAR evaluates follow-up inspections collected during the previous year to determine the performance in the previous inspection cycle. Due to the biennial nature of our Smog Check Program, inspections performed over three years ago may still affect an FPR score. For this reason, it can take considerably longer to achieve a "No Score" than to rehabilitate an FPR score with higher quality inspections.6.7 How does the FPR differ from the SVFR?
The Similar Vehicle Failure Rate (SVFR) compares the initial test failure rate for vehicles inspected at each station to the initial test failure rate for similar vehicles statewide. The FPR is similar in concept, however, it measures whether the actual vehicles certified by each station or inspector in the last cycle are passing in the current cycle at a higher or lower rate than expected in comparison to other similar vehicles inspected throughout the state.6.8 Is my station required to inspect poorly maintained vehicles?
No, stations are allowed to specialize in the type of vehicles that they inspect and repair, but must post the types of vehicles for which they offer services. Keep in mind that the FPR is based on comparing similar vehicles and conditions to all the other stations statewide. Vehicles with known maintenance problems can be directed to the Referee for inspection.6.9 I can't control what happens to a vehicle after it leaves my shop. Won't this affect my FPR score?
Stations and inspectors have little control over what consumers do to their vehicles once the vehicle leaves their station. For this reason, it would be incorrect to evaluate a particular station's performance based upon how one, or even a few, vehicles performed during the next Smog Check inspection. For example, some properly inspected vehicles with passing emissions levels will fall into disrepair before their next Smog Check inspection. Conversely, other improperly certified high emitting vehicles may get fixed to run properly in their next inspection cycle. For this reason, the FPR measure is calculated using large amounts of data. This way, similar events like those just mentioned will tend to average out for all the stations and inspectors. Instead of the "one-of-a-kind" events, the trends associated with proper inspections versus improper inspections will determine the FPR scores.6.10 My customer refused additional necessary repairs after their vehicle's emissions marginally passed Smog Check. Will this affect my FPR score and STAR certification?
The STAR Program does not consider after repair emissions levels in the evaluation any performance measures. In fact, repair data doesn't factor into the STAR Program evaluation at all. Still, the FPR measure under the STAR Program does consider whether or not vehicles certified by each station and inspector pass at a higher or lower rate than average in the next inspection cycle when compared to similar vehicles. Stations that make a habit of certifying vehicles with incomplete repairs (e.g., masking a fuel system problem with a new catalytic converter) may notice a lower FPR score in the long-term, but it is unlikely that this behavior alone would push the station's or inspector's FPR score to an unacceptable level. Stations who want to improve their FPR score and become STAR certified should primarily focus on performing accurate Smog Check inspections according to the Smog Check Manual. If this is done, the marginal vehicles needing additional repairs will be less of a factor in the overall FPR calculations.6.11 If the FPR evaluates how vehicles previously certified by stations and inspectors are performing in the current inspection cycle, how will new stations and inspectors be evaluated under this performance measure?
Smog Check stations and inspectors with low inspection volumes, including newly licensed stations and inspectors, will not receive an FPR score. An FPR score will only be generated once there is a sufficient volume of follow-up inspections performed on vehicles previously certified by that station or inspector.
In most cases, a station or inspector without an FPR score will not be an issue in determining a station's eligibility for the STAR Program. The only time that an inspector with an FPR of "No Score" may be an issue is when a STAR certified station has a low FPR score (less than 0.40). STAR stations with low FPR scores cannot have an inspector entered into their analyzer(s) with an FPR of "No Score."
Similarly, STAR certified stations without FPR scores cannot have an inspector entered into their analyzer(s) with a low FPR score (less than 0.40). For more information on the various FPR rules for determining STAR Program eligibility, refer to the FPR Eligibility Chart.6.12 Will an inspector's FPR score be affected if they go on an extended leave of absence?
The FPR measures whether vehicles certified previously by each station or inspector are passing in the current inspection cycle at a higher rate than average. Vehicles previously certified by higher performing stations and inspectors will tend to fail at a lower rate in the current inspection cycle.
Because the FPR reflects station and inspector performance on vehicles they previously certified, stations and inspectors do not have to be currently performing inspections in order to generate an FPR score. In addition, vehicles previously certified by a station or inspector do not have to be re-inspected by the same station and inspector in order for the vehicle to be evaluated under the FPR.
An inspector who takes a leave of absence for a period of time could still have an FPR score and that score could change based on the results in the current inspection cycle of vehicles the inspector previously certified. If, however, the inspector is on leave for a period of years, there will eventually be a point at which there are no longer any vehicles certified by the inspector to generate an FPR score. At that point, the inspector will cease to have an FPR score under the STAR Program. However, an inspector without an FPR score can still perform Smog Check inspections at a STAR station, provided that station has an FPR score of 0.40 or higher, or does not have an FPR score because it is either a new station or, similar to the inspector in this example, has an extremely low inspection volume.
FPR scores are updated every six months, in January and July.6.13 How can a station with a high FPR score have two inspectors with entirely different FPR scores?
First, it is important to remember that FPR scores are affected by both the quality of inspections performed and inspection volume for vehicles previously certified by both stations and inspectors.
For example, let's assume that two inspectors have been entered into the same station's analyzer(s) for years, and are not entered into the analyzer(s) of any other stations. Let's also assume that the two inspectors previously certified the same number of vehicles now being inspected in the current inspection cycle, but one of the inspectors performed higher quality inspections than the other inspector. In this case, it is fairly simple to understand that the higher performing inspector will have a better FPR score.
Now, let's consider another scenario for two inspectors entered into the same station's analyzer(s) for years at the same station. Let's also assume that both inspectors perform the same quality of work, but with one of the inspectors performing twice as many inspections as the other inspector. Which inspector will have the higher FPR score? The answer depends upon the quality of their identical performance. If their performance is above average, then the inspector with the higher inspection volume will have a higher FPR score because the statistical confidence of his or her higher performance will be greater with the larger sample size. Similarly, if their performance is below average, then the higher volume inspector will have the lower score because, again, the statistical confidence of his or her higher performance will be greater with the larger sample size.
For these reasons, one can't automatically assume that an inspector with a score of 0.90 performs higher quality inspections than an inspector with a score of 0.75. Instead, scores should be evaluated by whether or not they are above 0.50. Scores above 0.50 are trending better than average. Above average performance should be the goal for stations that want to participate in the STAR Program. Note that the FPR performance standard for inspectors entering a STAR station is 0.40, which is below the 0.50 level mentioned above. This lower limit was selected to allow inspectors near the standard the benefit of the doubt.6.14 Why is my FPR so low, even though my results for the STAR short-term measures look good?
The first reason why short-term results can be different from the FPR results has to do with the time frame over which the inspection results generated by the station or inspector are being evaluated. The short-term performance measures evaluate inspection performance based upon the most recent calendar quarter of data. The FPR, however, evaluates the inspection performance for each station or inspector in the previous inspection cycle. As a result, the short-term performance measures are evaluating performance over a different time frame than the FPR. Consequently, differences between short-term results and FPR scores could result from things such as station personnel changes or station policy changes.
Other reasons why the short-term results can differ from the FPR scores are based upon the nature of the measures themselves, as is shown below.
The short-term performance measures are designed to provide a simple litmus test of recent performance to evaluate stations (and the work done by their inspectors). It can be a great resource in helping a station identify problem areas and improve their overall performance in the Smog Check Program. For instance, if a station is failing to perform the fuel evaporative test on vehicles that should be tested, there is a short-term measure that can help a station identify that as an area needing improvement.
The short-term measures are not, however, comprehensive. For example, a station's inspectors can indicate in the analyzer that the ignition timing is adjustable for all vehicles tested at the station, and give that station a passing result on this performance measure. However, that does not mean that the station's inspectors actually performed the timing test, or that the timing test was performed accurately. Similarly, a station may have a low ASM restart rate, which would indicate that its inspectors don't appear to be trying to find ways to get vehicles that should otherwise fail to instead pass the test. At the same time, however, that station's inspectors may be routinely over-conditioning vehicles prior to starting the inspection. Neither the improper timing inspections, nor the habitual over-conditioning of vehicles, as well as other more serious violations, such as clean-piping, would be identified by the short-term measures.
For this reason, BAR felt it necessary to develop a more robust, long-term measure of station and inspector performance in the Smog Check Program. The result is FPR, a performance measure that evaluates both station and inspector behavior from inspection cycle to inspection cycle. Anything inspectors do to get a vehicle that should fail its Smog Check inspection to pass while still in a failing condition will start to affect their FPR score. Stations that want to score well on both the short-term measures and FPR will want to strive to always perform accurate Smog Check inspections.6.15 In order to maintain a high FPR score, should I refuse to inspect older, higher mileage vehicles?
No, even though Smog Check stations have the right to refuse to inspect vehicles they are uncomfortable testing. The reason is that all of the performance measures used in the STAR Program, including the FPR, compare the vehicles inspected by each station and inspector to similar vehicles inspected throughout the state. Therefore, if your station only inspects older, higher mileage vehicles, then the results for your station are only going to be compared to the statewide results for similar older, higher mileage vehicles. Because of this important factor, there is no more reason to be concerned about certifying an older vehicle with a carburetor than there is certifying a newer vehicle with fuel injection. What matters is that inspectors perform accurate inspections and properly fail vehicles that should fail, regardless of the vehicle age or mileage.6.16 What causes FPR scores to change?
Each time FPR scores are updated, some stations and inspectors may find that their FPR scores change. This will be more evident for stations and inspectors whose Smog Check inspection performance is closer to average. Others, whose behavior diverges more from average, will find their scores to be more stable, either at the high end or the low end of the performance scale. If a station owner or inspector wants consistently high FPR scores, then performing consistently high quality inspections is critical.
But, it is important to point out that sample size can affect STAR scores. The more vehicles previously certified by a station or inspector that are inspected in the current cycle, the easier it is to determine whether or not that station or inspector's performance is above or below average. This means that scores can change due to the number of vehicles being inspected in the current cycle. So, if two inspectors perform similarly and above average, the inspector with the higher inspection volume will have a higher score. On the other hand, if two inspectors perform similarly but below average, the inspector with the higher inspection volume will have a lower score. This feature can also affect individual inspectors. For example, if an inspector maintains a consistent level of above average performance with FPR scores of 0.60, that inspector's FPR scores would move closer to 1.00 with an increase in the number of inspection performed by the inspector.
It is also important to remember that inspector performance can and does change over time and this can affect both a station and inspector's FPR score. Station owners and inspectors need to reflect on what was happening two to three years ago to understand how these changes may be driving today's FPR scores. Were station owners or managers pushing high throughput at the expense of inspection quality? Were inspectors over-conditioning vehicles or driving them in the incorrect gear? Were inspectors taking the time to thoroughly visually inspect and functionally test all of the emission-related components as required by the Smog Check inspection procedures? When trying to understand FPR scores today, look back in time to see what was driving scores today.
Station FPR scores can also change based upon which inspectors were operating at the station at the time the vehicles were last certified and who was managing the station. Even though a station may take steps to address a problem where an inspector was performing improper inspections, the improper certifications issued by that inspector can affect that station's FPR score for some time.
It's also important to realize that accurate Smog Check inspections may take more time to perform. As a rule of thumb, a comprehensive inspection, including accurate determination of proper inspection procedures and thorough performance of the visual inspection and all applicable functional tests, will often take 20 or more minutes. These times may be reduced some if the inspector is experienced and has a lot of familiarity with the vehicle. Still, as evidenced by results on the STAR short-term performance measures, many inspectors are making more mistakes than they probably realize. When in doubt, slow down and perform the inspection correctly.Can't find the answer to your question? Please contact the BAR Industry Help Desk at email@example.com.