A store’s scorecard said two of its three service-desk staff needed to sell better. Its own records pointed somewhere else.
- Role
- The analysis: parsing the notes, the counts and the re-checks.
- Stage
- Analysis of one store’s exported records. No change was run.
- Outcome
- The recommendation narrowed, and one finding of mine was withdrawn. No result measured.
Select a person to read what their row suggests, and what it can’t prove.
| Person | On record | Said yes |
|---|---|---|
| Person A | 87 of 885 visits 9.83% | 63 of 87 on record 72.4% |
| Person B | 14 of 972 visits 1.44% | 14 of 14 on record 100.0% |
| Person C | 23 of 901 visits 2.55% | 20 of 23 on record 87.0% |
When B and C had a recommendation on record, customers said yes at least as often as for A. So why were there so few on record?
The situation
One store’s service desk, the three people who write up most of its paid visits, and 198 days of records. The scorecard ranked them plainly: about the same number of paid visits each (667 to 694), yet Person A sold 1.95 hours of work per visit, Person B 1.63 and Person C 1.39. Source: Report §01
Read off the scorecard, the answer to “who needs coaching, and on what?” is that B and C need to sell better. I wanted to know whether the records said the same thing.
Source and scopeOne store, three people, 198 days of records; how the counts were made
- store
- 1
- primary staff at the desk
- 3
- days of records
- 198
- written notes
- 13,710
- lines of work
- 14,357
Two populations, kept apart
2,034
paid visits on the store’s scorecard: A 667, B 694, C 673. Source: Report §01
2,758
visits behind the recommendation rates: A 885, B 972, C 901. Source: Report §03
The report doesn’t say why the second count is larger, so this page never adds, divides or swaps one for the other. Four other staff, with 3 to 14 visits each, were left out as too small to compare: this is one store, not a finding about every store. Source: Report footer
Reading the sources. “Report §03” is section 3 of the source report: my analysis of one store’s exported records, with each note split into sentences and matched by pattern. The report is private, so its numbers are quoted here, with its own contradictions noted where they matter. Names, the store and pay figures are left out.
What I found
Under every line of work sits a free-text note: the complaint, the cause, the fix. Split into sentences and matched by pattern, they held 157 visits with a written no that named the service refused. Set beside the sales, those noes give a number the scorecard doesn’t have: how often each person had a recommendation for one of three big repairs on record (an oil pump reseal, a timing belt service or a valve adjustment), and how often the customer said yes. That is the chart above. Source: Report §03
The widest gap was in how often a recommendation was on record, not in the yes. That is a pattern in the record, not a verdict on people. B’s 14 and C’s 23 recorded recommendations are small samples. The report’s 95% intervals for the three on-record rates don’t overlap. Those intervals treat each person’s visits as independent draws at one fixed rate, which a different schedule and car mix per person doesn’t guarantee; under that model they suggest the recorded rates differ. They can’t say why. Someone who recommends out loud and never writes down the no looks exactly like someone who never recommended, and the records don’t show which cars each person was given. So the records can’t show that selling skill doesn’t matter. They can show where to look first. Source: Report §03 Source: Report footer
Options I considered
Three interventions fit some of the numbers. There is no record of a meeting where they were weighed against each other, so read them as the candidate readings I worked through, each with what supports it and what is missing.
The decision and what I cut
The candidate that held up was the narrow one: make recommending the three big repairs, and writing down the customer’s answer, consistent. Before judging anyone’s skill, check how work and cars were handed out. The finding that taught me that second rule was one I had to withdraw.
The finding I withdrew Withdrawn finding
Person B’s hours sold per car, Period 1 to Period 6
2.89 to 1.65-43%
My first reading
B’s work collapsed and needed an urgent conversation.
Hours sold -1%
Cars assigned +74%
The first period was also lumpy: its median visit was 1.20 hours, and taking out its single largest visit drops it from 2.89 to 2.47. Source: Report §08
Hours per car has a bottom number the person doesn’t fully control, and the records don’t show how cars were assigned. Read in hours instead, the trends reorder: B flat, A down 17% on similar volume, C up 23%. None of that is a verdict either. It is where to look next: allocation and context, before anyone is told their work collapsed. Source: Report §08
What I cut from the recommendation:
- The finding that B’s work collapsed, above.
- Coaching on the scorecard measures that barely differ: lines of work per visit (7% apart), labor rate (4% apart), vehicle check-ups (88% to 94% for all three). Source: Report §12
- A ceiling estimate for near-empty visits, which the report itself labels a ceiling, not a forecast. It pulled attention from the main question.
What I built
What I built was the analysis itself, made inspectable: a method that turns free-text notes into a count, and the checks that caught my own mistakes. Here is the method, run on a visit written for this page. Switch the no from written to spoken and watch the recommendation disappear.
One visit through the method Synthetic example
This visitCounted: 1 recorded recommendation
Synthetic visit S-01 · written for this page
Line 1 · Basic servicePERFORMED A SERVICE PER SCHEDULE. RECOMMENDED +4 SERVICE (TIMING BELT) DUE TO AGE. CUSTOMER DECLINED TIMING BELT TODAY.
Line 2 · Brake checkPADS WITHIN SPEC. CUSTOMER REFUSED BRAKE FLUID SERVICE.
Line 3 · Road testROAD TESTED, NO CONCERNS. CUST DECLINED +4 SERVICE AGAIN.
1 · Match sentences
Keep a sentence if it says
DECLIN,REFUSorDENIEDand names a service or its maintenance code.3 of 7 sentences kept
2 · Service family
Tag each kept sentence: one of the three big repairs, or something else.
- Timing belt (+4 code): big repair
- Brake fluid (+7 code): other, not counted
- Timing belt (+4 code): big repair
3 · One per visit
Two noes on one visit for the same family still count once.
2 big-repair sentences → 1 visit
4 · Recorded recommendation
A visit counts if it sold one of the three, or holds a written no for one.
Counted: one recorded recommendation, declined.
The hard part wasn’t the pattern match. It was deciding what the count can mean. Sales come from the repair codes on each visit; noes come from the notes. Together they give a recorded-recommendation proxy: a floor on how often each person recommends, not a census of every conversation, and not a list of the customers who needed the work. Source: Report §13
82 of 92 checks against the store’s own report passed the first time. Source: Report §13
- Wrong line basis. I had counted every paid line of work; the store’s report counts only lines with a labor sale. Fixing it flipped a headline: C’s rate of finding a repair is 25.9%, not 30.1%, the lowest of the three.
- Stale figures. Four of C’s values still came from an earlier, one-period file; one was out by 87%.
- A ratio read as a trend. No check caught this one; a separate sanity test did. It is the withdrawn finding above.
The report puts the ten failures down to bugs of mine and lists the fixes. It doesn’t record a final run of all 92, so I don’t claim one.
What happened
Two things changed, both in the analysis. The proposed intervention narrowed from “sell better” to “recommend, and record the answer, consistently”, with allocation checked first. And a tempting conclusion about Person B was withdrawn.
Nothing in the material I have shows what happened next: no recorded change in sales, in customer satisfaction or in how anyone worked. The report did put money on the opportunity, in two separate models. Here they are as the report states them, with what each one is missing.
Model A Source-reported model
+$307,556 a year
Unit: modeled store profit, as the report labels it
- Assumes
- B and C sell oil pump reseals, timing belt services and valve adjustments as often as A, at the prices all three already charge. Scaled by 365/198.
- Also reported
- +$29,513 a year in staff commission, as a separate figure.
- Missing
- Per-repair counts and their visit totals, profit per job, and how commission was treated. The report gives the result as a fixed number; I could not reproduce it.
Source: Report §07 Source: Report §10
Model B Source-reported model
+$260,363 a year
Unit: labor sales, not profit
- Assumes
- B and C have a recommendation on record as often as A (87 of 885 visits), with yes held down to A’s 72%. Scaled by 365/198.
- Missing
- The average labor sale per sold big repair, and how visits with several of them were counted. The dollars can’t be rebuilt; the counts can, below.
Source: Report §03 Source: Report §12
Different assumptions, different units: neither adds to the other, and neither happened. Both are the report’s opportunity estimates, not revenue anyone earned.
Model B’s counts, rebuilt Planning scenario
recommendations = visits × 9.83% (87 ÷ 885) · yeses = recommendations × yes-rate
| Person | Visits | Recommendations | Yeses |
|---|---|---|---|
| Person B | 972 | 14 on record 95.55 scenario ≈ 96 | 14 on record 68.80 scenario ≈ 69 |
| Person C | 901 | 23 on record 88.57 scenario ≈ 89 | 20 on record 63.77 scenario ≈ 64 |
A planning scenario over the same 198 days: counts, kept fractional, rounded only for reading. It is not a forecast, and not a count of customers who needed the repair.
What I'd change now
I would lead with the uncertainty instead of the headline. The report’s own top line said nobody there needed sales training; the records support a narrower claim than that. Before a recommendation rate goes on a wall, I would run this.
Not yet run Proposed validation
- Define who was eligible. Which visits could have needed one of the three repairs: by age, mileage once it is captured, and service history.
- Reconcile the two visit counts. Explain the gap between 2,034 scorecard visits and 2,758 in the rate denominators before comparing anyone.
- Audit the record. Sample visits with and without a written recommendation and check them against what was actually said.
- Control for allocation. Compare people within the same day, car mix, Saturday share and warranty load.
- Measure one intervention. Change one thing, such as a consistent prompt to recommend and record the answer, and compare against a stated baseline.
The method would stay: read the notes, count what is on record, and say plainly what the count can’t see.