Senior Data Scientist, EngThrive Insights - CoreAI

MicrosoftRedmond, WashingtonOn-siteFull-timeSenior, 5–8 yearsListed 13 minutes ago

Apply now

About this role

<b>Overview</b><br><div><p><span><span>Join the team </span><span>defining</span><span> what AI-first software engineering looks like at Microsoft. </span><span>EngThrive</span><span> Insights</span><span> build</span><span>s</span><span> data products that help tens of thousands of </span><span>Microsoft </span><span>engineers </span><span>to </span><span>work more </span><span>productiv</span><span>el</span><span>y with AI</span><span>.  </span><span>We collect thousands of engineering signals daily from across the entire SDLC. </span><span>We</span><span> then</span><span> partner with all levels of engineering – from executives to ICs – to translate the data into tools and programs t</span><span>hat</span><span> maximize AI-fu</span><span>e</span><span>led productivity</span><span>.  </span></span><span> </span></p></div><div><p><span> </span></p></div><div><p><span><span>In this role, </span><span>you'll</span><span> be a measurement expert for </span><span>EngThrive</span><span>: designing metrics that developers and leaders trust, combining telemetry with survey data, and running the causal analyses that show what </span><span>actually makes</span><span> engineers more productive with AI. </span><span>You'll</span><span> publish and present your findings across Microsoft and</span><span> with</span><span> the </span><span>broader </span><span>research community. </span><span>A</span><span>nd </span><span>you</span><span>’ll</span><span> work </span><span>hand-in</span><span>-</span><span>hand</span><span> with </span><span>e</span><span>ng</span><span>ineering </span><span>teams to </span><span>turn your insight</span><span>s into action</span><span>. </span></span><span> </span></p></div><div><p><span> </span></p></div><div><p><span> </span></p></div><br><br><b>Responsibilities</b><br><div><ul style="list-style-type:disc"><li><p><span>Partners with engineering teams and leaders to understand how they think about productivity and developer experience in the context of AI. Translates their questions into testable hypotheses, metric definitions, and research studies. </span><span> </span></p></li></ul></div><div><ul style="list-style-type:disc"><li><p><span>Designs, validates, and maintains developer productivity and experience metrics. Balances system telemetry with perceptual measures, documents clear definitions, and builds in guardrails that ensure privacy-by-design.</span><span> </span></p></li></ul></div><div><ul style="list-style-type:disc"><li><p><span>Designs and analyzes large-scale developer surveys, including the Engineering Experience Survey. Applies sound questionnaire design, sampling, and reliability and validity testing. </span><span> </span></p></li></ul></div><div><ul style="list-style-type:disc"><li><p><span>Applies causal inference, experimentation, and longitudinal methods to estimate the impact of AI tools, engineering-system investments, and team practices on productivity, quality, and satisfaction. Communicates effect sizes and uncertainty clearly. </span><span> </span></p></li></ul></div><div><ul style="list-style-type:disc"><li><p><span>Uses AI tools across the data science workflow, validates every AI-generated result, and remains accountable for the final output. </span><span> </span></p></li></ul></div><div><ul style="list-style-type:disc"><li><p><span>Contributes to thought leadership through papers talks. Collaborates with Microsoft Research and the broader developer research community to advance the frontier of developer productivity. </span><span> </span></p></li></ul></div><br><br><b>Qualifications</b><br><div><p><strong>Required Qualifications</strong></p></div><div><ul><li>Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)<ul><li>OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)</li><li>OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)</li><li>OR equivalent experience. </li></ul></li></ul></div><div><p><strong>Other Requirements</strong></p><p>Ability to meet Microsoft, customer and/or government security screening requirements is required for this role. These requirements include, but are not limited to, the following specialized security screenings:</p><ul><li>Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.</li></ul><p><strong>Preferred Qualifications</strong></p></div><div><ul style="list-style-type: disc;"><li><p>5+ years of experience measuring or researching developer productivity, developer experience, or engineering effectiveness (e.g., on an engineering productivity, developer experience, or DevOps research team, or at a developer analytics company). </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Experience applying developer productivity frameworks such as SPACE, DORA, DevEx, or DX to define and operationalize metrics. </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Experience with causal inference on observational data (e.g., difference-in-differences, panel or fixed-effects models, propensity methods) and design of controlled experiments. </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Experience designing and analyzing large-scale surveys, including psychometric validation (e.g., reliability analysis, factor analysis).  </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Experience combining system telemetry (e.g., pull request, build, CI/CD, and AI tool usage data) with self-reported measures. </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Experience measuring the adoption and impact of AI coding tools or agents on software developers. </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Proficiency in Python or R and analytical query languages such as SQL or KQL. </p></li></ul></div><div><ul style="list-style-type: disc;"><li><p>Demonstrated use of AI in a daily data science workflow. </p></li></ul></div> <br><br><p>Data Science IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 - $261,000 per year. </p><p></p> <p>Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:<br><a href="https://careers.microsoft.com/us/en/us-corporate-pay">https://careers.microsoft.com/us/en/us-corporate-pay</a></p><br><p>This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.</p><br><hr><br><p>Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about <a href="https://careers.microsoft.com/v2/global/en/accessibility.html"><b><u>requesting accommodations.</u></b></a></p>