After analyzing 200,000 anonymized conversations with Bing Copilot, Microsoft researchers have produced one of the clearest, most data-driven snapshots yet of how generative AI is seeping into real jobs. The study, titled Working with AI: Measuring the Occupational Implications of Generative AI, moves beyond theoretical projections by grounding its findings in actual user behavior. The results are sobering for knowledge workers: language- and information-heavy roles show the highest overlap with today’s AI capabilities, while physically demanding, hands-on occupations remain largely insulated.
The research, now available as a preprint on arXiv and published by Microsoft Research, was conducted by Kiran Tomlinson, Sonia Jaffe, Will Wang, Scott Counts, and Siddharth Suri. They tapped into two distinct samples of Bing Copilot conversations collected between January 1 and September 30, 2024. One sample of roughly 100,000 conversations was drawn uniformly from all U.S. Copilot users; the other 100,000 was enriched with conversations where users explicitly gave thumbs-up or thumbs-down feedback. Both datasets were anonymized and scrubbed of personally identifiable information under Microsoft Institutional Review Board oversight.
A New Way to Measure AI’s Impact on Work
Instead of asking what AI could do, the team examined what people are actually asking Copilot to do. Every conversation was run through a GPT-4o-based classification pipeline that labeled it with matching Intermediate Work Activities (IWAs) from the U.S. Department of Labor’s ONET taxonomy. ONET defines 332 IWAs—granular task descriptors like “analyzing data or information” or “communicating with persons outside organization”—that together form the building blocks of every occupation.
Crucially, the classifiers distinguished between the user’s goal (what the person wanted to accomplish) and the AI’s action (what Copilot actually performed). This split allowed the researchers to separate augmentation—where the AI assists a human on a task—from automation, where the AI completes the task independently. The approach mirrors the real-world nuance that Copilot often drafts an email a user will later edit, rather than autonomously firing it off.
From these labeled conversations, the team computed an AI applicability score for each occupation. The score aggregates three metrics:
- Coverage: How many of an occupation’s IWAs appear frequently enough in Copilot conversations to be meaningful.
- Completion: How often Copilot’s output was judged successful, using either user feedback or an automated classifier.
- Scope: Whether Copilot’s involvement covered a moderate or large share of the work activity.
A high score means that Copilot is already being used often, successfully, and across a significant chunk of what the job entails. A low score signals that the job’s real-world activities rarely intersect with the kinds of tasks Copilot handles well.
The Jobs AI Is Already Reshaping
The top of the ranking reads like a who’s who of desk-bound, language-centric roles. Interpreters and translators came out on top, followed by historians, passenger attendants, sales representatives (services), writers and authors, customer service representatives, CNC tool programmers, telephone operators, ticket agents and travel clerks, and broadcast announcers and radio DJs. The full top 40 is dominated by occupations that hinge on producing, translating, or summarizing information—think technical writers, market researchers, and clerks.
“These are jobs where a lot of the day-to-day work involves drafting, translating, summarizing, or retrieving knowledge,” the authors note. Large language models excel at exactly those tasks, and the Copilot logs confirm that users are leaning heavily on the tool for them. Industry watchers have seized on the list as a practical guide for workforce planning. GeekWire and Investopedia, among others, reported the findings as a clear signal that sales, marketing, and customer communication tasks are already being automated at scale.
At the opposite end, the least-affected occupations are those demanding physical presence, dexterity, and hands-on problem-solving. Phlebotomists, nursing assistants, hazardous materials removal workers, tire repairers, dishwashers, roofers, pile driver operators, dredge operators, and water treatment plant operators populate the bottom 40. These roles rely on embodied judgment, complex manual coordination, or safety-critical operations that today’s text-based AI cannot replicate.
Why Real Usage Data Changes the Game
Most earlier studies estimated AI exposure by comparing job task descriptions to LLM capabilities in the abstract. That approach can overstate overlap because it assumes both that AI can perform a task and that workers would choose to hand it off. Microsoft’s study anchors everything in observed behavior: what users actually ask Copilot to do and whether the AI delivers. This shift from theoretical possibility to verified practice is a major methodological leap.
It also aligns with what companies are already experiencing. Microsoft itself disclosed that AI-driven efficiencies in its contact centers saved more than $500 million in the past year, a figure reported by Reuters and ITPro. While these savings are corporate disclosures and not independent audits, they underscore that the operational impact is real and immediate. When paired with workforce restructuring—Microsoft has also cut jobs in some areas—the numbers paint a picture of rapid organizational change.
Strengths and Limitations of the Research
The study’s strengths are substantial. It leverages a massive, real-world dataset that captures voluntary, unscripted AI use across millions of tasks. The O*NET mapping provides granular, standardized job profiles, and the separation of user goal and AI action adds analytical depth. Validation against human annotators and the use of multiple success signals—thumbs feedback plus automated classifiers—help mitigate bias.
However, several caveats warrant caution. First, the data is entirely U.S.-based and drawn from Bing Copilot, which is tightly integrated with search. This likely inflates the prevalence of information-gathering tasks relative to other AI assistants or enterprise tools. The sample also skews toward early adopters and may not represent the broader workforce. Extrapolating the rankings to other countries or AI platforms should be done carefully.
Second, the AI applicability score measures technical overlap, not job loss. The study authors explicitly warn against conflating high applicability with inevitable displacement. Historical parallels—like ATMs and bank tellers—show that productivity gains can sometimes expand employment in unexpected ways. The downstream effects depend on how firms restructure work, invest in new roles, and adapt business models.
Third, the study is blind to non-text AI. Advances in computer vision, robotics, and embodied AI could eventually erode the current insulation of manual jobs—a factor not captured by Copilot chat logs. Generative AI also brings error modes: hallucinations, factual inaccuracies, and biases that may raise, rather than reduce, the human cost of oversight and correction. A thumbs-up doesn’t guarantee flawless output.
What the Findings Mean for Workers, Employers, and Policymakers
For workers, the message is urgent but not hopeless. Tasks that resist automation—interpersonal negotiation, deep domain expertise, complex project management, and creative leadership—remain valuable. Deliberate upskilling in prompt literacy, AI oversight, data interpretation, and cross-disciplinary skills can help individuals stay ahead. Using AI for routine drudgery frees time for higher-value work, but that shift won’t happen automatically; it requires conscious job redesign and personal initiative.
Employers should audit roles at the task level, not just the job title. Identify where AI can safely absorb repetitive load and redeploy talent toward strategic activities. Design reskilling pathways and internal mobility programs to avoid sudden workforce shocks. Governance is critical: set clear rules for verification, data privacy, and human review before letting AI loose on customer-facing or regulated processes.
Policymakers must monitor labor market signals—job posting trends, wage shifts, occupational transitions—to spot early dislocations. Public investment in retraining, portable benefits, and bridging programs can soften the blow of rapid adoption. Regulation should be risk-based, demanding stronger transparency, auditability, and human-in-the-loop safeguards where AI decisions affect safety, legal rights, or economic livelihood.
Practical Steps for Organizations Deploying Generative AI
Drawing from the study’s insights, organizations should start with a measured, evidence-based approach:
- Map Tasks, Not Just Roles: Decompose jobs into IWAs and prioritize automation where it is safe, verifiable, and genuinely reduces workload without compromising quality.
- Pilot with Metrics: Track time saved, error rates, verification costs, and downstream impacts on staffing before scaling any AI deployment.
- Set Human Verification Thresholds: For high-risk outputs—legal documents, medical summaries, safety instructions—mandate human review and assign clear accountability.
- Invest in Retraining: Create clear career paths for employees whose tasks are automated, moving them into oversight, exception handling, or new roles that leverage their institutional knowledge.
- Publish Transparency Reports: Build trust by disclosing how AI is used in customer interactions and employment decisions, along with audits of accuracy and bias.
The Bottom Line
Microsoft’s Copilot-based study provides the most concrete early map yet of where generative AI is already proving useful—and where it isn’t. By anchoring its analysis in 200,000 real conversations, the research cuts through speculation to deliver operationally relevant signals. Language- and information-centric work shows the highest present-day overlap with LLM capabilities; manual, embodied roles remain the least affected.
Yet the most consequential unknowns remain organizational and political. The study stops short of predicting headcount changes because those outcomes hinge on corporate decisions, public policy, and worker response. Microsoft’s own reported $500 million in contact-center savings shows that decisions are being made quickly—and often behind closed doors. The net benefit for workers will depend on deliberate adaptation, continuous reskilling, and robust governance. AI is not an unstoppable force; how we choose to deploy it is still very much in our hands.