Your direct reports watch you every day. Your boss does not. They see whether you actually delegate, whether you follow through on one-on-ones, whether you stay calm when a project slips. That daily signal is the cheapest corroborating evidence of behavior change you will ever collect, and most organizations never ask for it.
Upward feedback for a manager works when three conditions hold: the questions describe behavior, the reporting is aggregated and anonymous, and nothing about the manager’s pay depends on the result. Miss any of them and you get a popularity contest, or a scored survey nobody answers honestly.
This piece covers the mechanics. Why direct reports see what a boss cannot, and what upward feedback can and cannot prove. How to write behavioral items instead of sentiment items. The anonymity and aggregation rules that keep the signal honest, and the pay firewall that keeps compensation out of it. How a short anonymous pulse differs from a manager effectiveness index. And what to do with the signal once you have it.
Why Direct Reports See What a Boss Cannot
This is for HR, L&D and executives running a manager cohort at a mid-market or enterprise company, or for anyone working out what a leadership academy should actually do before the first session.
It is not for an individual manager buying Tandem Academy at $1,000 a year or $99 a month, who needs what an AI coach can and cannot do instead. Anonymity, aggregation and a pay firewall hold only inside a managed group where someone other than the manager controls the data.
A manager’s boss sees the visible half: the one-on-ones that happened, the deadline met, the escalation handled. The boss does not see the one-on-one rescheduled six times, the question answered with “figure it out”, or the credit taken in a team meeting.
Direct reports sit inside the daily unit of behavior: how work is assigned, how questions get answered, how mistakes are handled, how credit and blame are distributed. That is the behavior you are trying to measure, and it happens where the boss is not.
Smither, London and Reilly (2005) report that direct report, peer and supervisor ratings relate to other effectiveness measures, and that raters including direct reports conceptualize performance in similar terms.
Upward Feedback for a Manager: What It Can Prove, and What It Cannot
Set the claim precisely. Upward feedback is corroborating evidence of behavior change. It is not proof of leadership quality, and it is not a ranking of managers. Treating it as either loses the mechanism.
The evidence supports direction, not magnitude. Direct report ratings relate to broader effectiveness measures, and Smither and Walker (2004), reported in Smither, London and Reilly (2005), found managers whose direct reports wrote more favorable narrative comments also got more favorable annual reviews from their supervisors.
State the limit before the first pulse
A meta-analysis of 24 longitudinal studies found improvement in direct report, peer and supervisor ratings over time to be “generally small.” The authors advise practitioners not to expect large, widespread improvement after multisource feedback (Smither et al., 2005).
Change is conditional, and the conditions are yours to design
Improvement is more likely when feedback signals that change is necessary, the recipient has a positive feedback orientation, perceives a need to change, reacts positively, believes change is feasible, sets appropriate goals and acts on them (Smither et al., 2005). A pulse that produces a score with no goal-setting step attached is collecting data, not causing change.
The review dates from 2005. Design the pulse as a small measurement repeated at fixed points, treat one cycle as directional, and verify effect sizes in the source article before quoting any.
Write Behavioral Items, Not Sentiment Items
The distinction decides whether you have evidence or a mood reading.
“Did your manager give you what you needed to do this task well?” is a behavior question.
“Do you like working for your manager?” is a sentiment question. Ask it and you have pulled a popularity contest into your data set.
Test each item against four filters: it describes an observable act; two trained observers could agree it happened in the last thirty days; it is answerable from memory, not inference; and the manager could do something differently tomorrow if the answer is no.
Draw stems from behavior dimensions, not relationship adjectives:
- Clear expectations: “Did your manager tell you what good work looks like before you started a task?”
- Coaching conversations: “Did you talk about your development outside your annual review?”
- Delegation: “Were you given a decision you were expected to make?”
- Accountability: “When a commitment slipped, did you hear about it directly?”
- Feedback and trust: “Did you get feedback specific enough to act on?”
Use frequency and occurrence wording, not agreement scales. “How often” and “did this happen this period” produce countable answers. “Strongly agree” absorbs personality, tenure and how the quarter went.
Include a “did not occur this period” option. Without it, a manager who gave no feedback and a manager whose team had nothing to report score the same, the ambiguity the item exists to remove. Treat these as design choices, not settled findings.
Comparability is the payoff. Sentiment items and narrative comments move with team mood and cannot be compared across cycles. Behaviorally anchored items can be, which is the point of asking at three fixed points. You get a number. You get it three times.
Anonymity and Aggregation: The Mechanics That Make the Signal Honest
Behavioral items make cycles comparable. Reporting rules make them honest. Set four before the first pulse.
Set a minimum reporting group. Below that number, no result is reported for that manager. On a two-person team, an average is not anonymous. It is a named opinion with a decimal point.
Write a suppression rule and apply it without exception. If respondents in a cycle fall below the threshold, suppress that manager’s result entirely. Do not flag it as low participation at the manager level. Report the count of suppressed managers at cohort level instead. Flagging invites the manager to work out who did not answer.
Collect through one point the manager does not control. Responses go to an independent party or a system the manager cannot log into. Not to the manager, not to the manager’s boss, not a shared inbox.
Report aggregates only. Show group means and distributions across cycles. No individual responses. Do not release free-text comments a manager could trace by content or timing. If you collect comments, release them only when the group is at or above the threshold, and edit out identifying detail first.
Roll results up a level where groups are small, and hold publication until the pulse has closed so early answers cannot be discussed while other people are still answering.
These thresholds are a design convention, not a research finding. No source here establishes a correct number. Choose one, publish it, and keep it identical cycle to cycle so the comparison holds and the reason for a suppressed result is never personal.
The Pay Firewall: Nothing Rides on the Result
The suppression rule protects the individual response. The pay firewall protects the whole mechanism.
It answers a specific finding. Ostroff, Atwater and Feinberg (2004), reported in Smither et al. (2005), found that subordinate and peer ratings show a small but statistically significant positive relationship with recipients’ compensation and organizational level. The link already exists. Do not build a mechanism that makes it stronger.
The rule: nothing in a manager’s compensation, bonus, performance rating or promotion decision may reference the upward pulse. Not averaged in. Not weighted at ten percent. Not considered alongside other inputs. Adjacency is enough to change an answer.
Publish the firewall in writing before the first pulse, in the same document that states the suppression threshold, so managers and their reports read the same words.
Keep the data separate. Pulse data lives outside the performance management system and is never exported into it. Once it is in the system, it is in a promotion file.
Separate the audiences. HR or L&D own the cohort aggregate. The manager’s boss sees the Manager Effectiveness Index score and the pulse trend. Individual comments stay with the function collecting them.
Name the consequence of breaking it. One cycle in which a manager can infer that a low score cost them something ends candid responses in every later cycle, and the mechanism cannot be repaired afterward. The firewall is the design choice that makes the pulse evidence instead of a survey nobody trusts. The same audiences need to understand the pay firewall for the same reason.
A Short Upward Pulse Versus the Manager Effectiveness Index
They measure different halves of the same question and cannot substitute for one another.
The Manager Effectiveness Index covers fifteen behaviors across five dimensions: clear expectations, coaching conversations, delegation, accountability, feedback and trust. The manager’s own boss scores it before, at the end, and ninety days later. Three fixed points make before and after comparable.
The upward pulse covers five to eight behavioral items, scored by direct reports, run at the same points or more, and reported only in aggregate under the suppression rule. It measures daily behavior; the index measures what the boss can see. Neither sees what the other sees.
Use the pulse to corroborate, not replace. If the index improved and the pulse did not move, treat the change as unverified and check the conditions: did the manager set a goal, did coaching continue through ninety days, were the items identical in both cycles.
Enterprise cohorts carry the Ninety-Day Behavior Change Guarantee: if the index score does not improve and conditions were met, Tandem runs another coaching cycle at no charge. It attaches to the index, not the pulse, and covers enterprise cohorts only. Tandem Academy carries no guarantee.
Keep the artifacts distinct: one boss-scored index with a guarantee, one aggregate pulse with a suppression rule. Enterprise pricing lives on the cohort page; Academy is $1,000 a year or $99 a month.
What to Do With the Signal
The two artifacts are built. Now decide what happens to the readings. Work in this order, and set every rule before the first cycle rather than after the first uncomfortable result.
- Write the five to eight behavioral items first. Test each against one question: could two observers agree it happened in the last thirty days? Drop anything that reads like a relationship question. Item wording is what separates corroboration from a mood reading.
- Publish the suppression rule and pay firewall in one document before the first cycle, see the rules above.
- Run the pulse at the end of the training and again ninety days later. Match the same points as the boss-scored Manager Effectiveness Index so the two readings can be compared.
- Report the aggregate to sponsors with the limit stated up front. Across 24 longitudinal studies, improvement after multisource feedback was “generally small” (Smither, London and Reilly, 2005). Small corroborated movement in behavior is a real result. An inflated claim is not one you can defend in the next cycle.
Conclusion
Upward feedback earns trust when the rules come first. Write behavioral items, fix the suppression threshold and pay firewall before anyone answers, and report aggregate movement honestly to sponsors. The signal is real but small; state that limit up front rather than promising a transformation the next cycle cannot defend.