Say It, Done: AI Agents in Social Science Research

From chatbots to agents that do the work

Minghong Shen · Postdoctoral Fellow, NYU Shanghai

Workshop on AI in Social Science Research

July 22, 2026

Opening

A question to start

How much of your research time goes to the grunt work:
literature, filing, tables, figures, drafting?

Any work you have to repeat is grunt work.

Don’t answer out loud. Just let the question sit for a few seconds. Most academics underestimate how much of their time is logistics, and overestimate how much expertise is required to delegate it to AI.

A brief history of AI

The Claude Code timeline

METR: the length of tasks AI can handle doubles roughly every 7 months, from 1 hour in early 2025 to 8 hours by late 2026

The top timeline shows the general AI evolution: chatbots → tool use → early agents → agent diffusion. The bottom timeline zooms into Claude Code specifically: research preview in Feb 2025 with Claude 3.7 Sonnet; GA in May 2025 alongside Claude 4; went viral during Winter 2025-26 as non-programmers discovered “vibe coding”; and by 2026 it became a serious research tool adopted by social scientists. The METR footnote quantifies the slope: the duration of tasks AI can autonomously complete has been doubling roughly every 7 months.

An experiment inside Anthropic

Automated Weak-to-Strong Researcher (AAR): Anthropic let Claude Opus 4.6 propose hypotheses, design experiments, and iterate on a real AI research problem on its own. (April 2026)

Human researchers

Two Anthropic authors

• Tuned 4 representative methods
• Took 7 days
• Best PGR = 0.23

Agent team

9 Opus 4.6 instances in parallel

• Proposed / designed / iterated on their own
• Took 5 days
• PGR = 0.97 (1.0 is the ceiling)

About 800 agent-hours and roughly $18,000 of compute: on a real AI research task, the agents clearly beat the lab’s own human researchers.

The AAR experiment was published on Anthropic’s Alignment Science blog in spring 2026. Two researchers spent 7 days tuning four methods and got PGR=0.23. Nine parallel Opus 4.6 agents, running 5 days (~800 hours, ~USD 18,000), got PGR=0.97. The model lab’s own researchers outperformed on their own benchmark by their own agents.

This is a story about efficiency

180 days Traditional research

60 days With a chatbot

20 days With AI agents

Benchmark: one high-quality study by one professor, who runs 2–3 projects at the same time.

The longer a project takes, the more an agent tends to matter.

This is the claim. Everything else in this talk explains why it’s true. 180 days → 60 days → 20 days. The jump from chatbots to agents is the bigger one, and it’s the one most people haven’t made yet.

No need to keep climbing

Agents run on computer skills, yet they are about to make most computer skills obsolete.

Most sociologists use Stata or R. The traditional path says learn Python next, maybe Julia for speed. Agents make that path obsolete. You don’t need to climb, you need to jump sideways. Natural language is the interface; clear thinking is the skill.

Meet the agent

A map of AI categories

🏢 AI labs

OpenAI
Anthropic
Google
DeepSeek
Moonshot

💬 Chatbots

ChatGPT
Claude
Gemini
DeepSeek
Kimi

🔧 AI apps

NotebookLM
Gamma
Elicit
Consensus
Notion AI

💻 IDEs

VS Code
Cursor
Windsurf
Antigravity
Zed

🤖 Terminal agents

Claude Code
Codex
Gemini CLI
OpenCode
Qwen Code

⚙️ Agent internals

Memory
Skill
Agent
Hook
MCP

Today’s focus: terminal agents, the most capable AI category right now. The internals on the right are the secret of their power.

This is a landscape map. Most people conflate all of these – they think ChatGPT and Claude Code are the same thing. They’re not. Chatbots answer questions. AI apps do specific tasks in a web UI. IDE-integrated AI helps you code inline. Terminal agents are different: they have access to your file system, they can run commands, and they can orchestrate multi-step workflows. The right column – hooks, skills, sub-agents, MCP – is what makes them programmable and composable. Today we focus on column 5 (terminal agents), and we’ll touch on column 6 to understand why agents compound over time.

Try it: what happens inside a Transformer?

Georgia Tech Polo Club · Transformer Explainer (needs internet) · If it won’t load, visit https://poloclub.github.io/transformer-explainer/

Live interactive demo, embedded directly in the slide (the site allows framing – no X-Frame-Options). Requires internet on the presenting machine; if offline, skip to the link in the footnote. Let the audience drive: type a short sentence, press generate, then expand the layers to watch embeddings → attention heads → MLP → softmax over the next-token distribution. The point for this talk: an LLM is not magic and not a database – it is a stack of attention and feed-forward layers turning text into probability distributions over the next token. Everything an Agent does (retrieval, tool use, planning) is scaffolding built around this core. Two minutes here saves ten minutes of confusion later about hallucination and context windows.

Pre-training vs. post-training

Base model · only continues text

Base Model

>▌

Chat model · actually answers

Chat Model

>▌

Ouyang et al. (2022). InstructGPT, NeurIPS · DeepSeek-AI (2025). DeepSeek-R1 · Raschka (2024). New LLM Pre-training and Post-training Paradigms

Pre-training = next-token prediction over raw web/books (trillions of tokens, thousands of GPUs, months) – self-supervised, no labels. The slide opens on the pre-training lane only; ONE CLICK overlays the full diagram revealing the post-training lane and the dashed “start from the base model” link. Meanwhile the two demo windows below type the same question in parallel: the base model just continues the text (bold output - it rambles more questions), the chat model actually answers. Post-training starts from that base model: SFT on human-written instruction/answer pairs (InstructGPT used ~13k labeler-written prompts), then preference alignment – RLHF (reward model from human rankings + PPO, the InstructGPT/ChatGPT recipe), DPO (skips the reward model), or RL with verifiable rewards for reasoning (DeepSeek-R1’s GRPO). Orders of magnitude cheaper: thousands-to-millions of examples, days not months. Concrete lineages: GPT-3 → InstructGPT → ChatGPT; Llama base → Llama-Instruct; DeepSeek-V3 base → R1. Why the audience should care: everything they touch – chatbots, agents – is a post-trained model. Instruction following, refusals, “as an AI” hedging all come from post-training, not pre-training. And fine-tuning an open model on your own corpus is the same two-step idea at lab scale.

The terminal: texting your whole computer

Stata Command window

Type a command → act on the data

Stata/MP · Command

.▌

.▌

.▌

The terminal

Type a command → act on the whole computer

pwd where am I · ls what’s here · cd go there

Terminal

~ $▌

~/research $▌

~/research $▌

~/research/data $▌

These are Mac/Linux commands; Windows PowerShell is almost identical (ls and cd work in both).

Two rows, same interaction pattern. Top: the Stata command window they already know - and the punchline is that Stata itself speaks shell: pwd prints the working directory, cd changes it (and echoes the new path, watch for that), then summarize answers a question about the data. So the conversation partner in Stata is the DATASET, but it can already do basic file navigation. Bottom: the terminal - same pattern, same commands, but the conversation partner is the WHOLE COMPUTER. Walk it: pwd answers “where am I”, ls answers “what is here”, cd moves into a folder - and the prompt changes from ~/research to ~/research/data, same job as Stata’s echo. Reassure them: nobody memorizes hundreds of commands; pwd/ls/cd plus one or two more cover 90% of daily use - and they effectively already know three of them from Stata. Everything is text, so everything can be scripted, logged, and repeated - which is exactly why agents love the terminal.

CLI tools = command + arguments; press Enter and it runs

Most tools have no graphical interface, just a command name:

git · an undo button for your code

Terminal

~/research $▌

~/research $▌

~/research $▌

~/research $▌

~/research $▌

stata · run a do-file from the terminal, no GUI needed

Terminal

~/research $▌

~/research $▌

~/research $▌

The stata command must be on your PATH first (a one-time setup)

The AI tools you’ll use are just another command name like these.

Generalize from the built-in commands: a CLI tool is just a program whose name you type, followed by arguments. Two examples side by side. Left, git as the “undo button” for their do-files: commit -am stages AND snapshots in one step; log –oneline lists the version history; then the visible payoff - head -1 shows the do-file’s first line (Revised: sampling weights added) BEFORE, checkout – analysis.do runs silently (checkout prints nothing - silence means success), and head -1 again shows the file is now the OLD version (First pass: baseline OLS). The do-file’s own content changing is what proves the rollback, not any git output. Right, the punchline for Stata users: Stata itself ships a terminal tool - stata -b do analysis.do runs the do-file in batch mode WITHOUT opening the GUI, writes results to analysis.log, and cat shows the log. Caveat to state plainly: the stata command only works after adding Stata’s install folder to PATH (one-time setup; on Windows the exe may be StataMP-64). This exact command is what the Claude Code demo on the next slide calls - watch for it. The purple box is the pivot: AI tools are installed and launched the same way.

Claude Code is just another terminal program

Terminal

~/research $▌

✻ Welcome to Claude Code · Opus 4.7

>▌

⏺ Let me check which variables are in the data first…

⏺ Bash(stata -b do check_vars.do) ✓

⏺ Found income, urban, age, educ, and more. Writing the script and running it…

⏺ Write(analysis.do) → Bash(stata -b do analysis.do) ✓

⏺ Urban households average $71,200 vs. $54,800 rural; regression table saved to results.md

>▌

⏺ Write(plot.do) → Bash(stata -b do plot.do) ✓

⏺ Chart saved as income_by_urban.png (urban–rural gap p < 0.001)

The only difference: you give orders in plain language.

Interface simplified; real versions differ slightly · Codex, Gemini CLI, and others work the same way: codex and gemini are each just a command name

The demystification slide. Type claude at the same prompt where they just typed pwd, git and quarto - that is the whole launch procedure. Then watch a full two-turn episode: the user asks in plain English; Claude inspects the data, WRITES a do-file, RUNS it through Stata’s own batch mode, reports the numbers; the user follows up with a plotting request and it writes and runs a second script. Point at the Bash/Write lines: the agent is using the very skills from the previous two slides - the terminal - on your behalf, and it loops until the job is done. The numbers are illustrative, not real GSS results. Closing frame: terminal agent = a terminal program whose command language is natural language. That is why the next section’s capabilities (reading files, running scripts, fixing its own errors) are not magic but consequences of living in the terminal.

What makes an agent different

The whole picture

It sees your scripts, data, and drafts at once. No more pasting snippets.

Self-debugging, end to end

It fixes bugs directly, with no hand-holding.

It doesn’t just answer questions; it gets the work done. Automate the grunt work and spend your time on the thinking.

How does an agent find what it needs?

An agent never loads the whole tree. It picks only the leaves it needs.

Purple = loaded, grey = skipped. The agent keeps only relevant knowledge in its “working memory”, which saves a lot of context.

Three-layer tree search, not a dump-everything approach. Layer 1: the user’s question arrives. Layer 2: the agent scans broad categories – do I need tools? skills? memories? Layer 3: within each relevant branch, it loads only the specific items that match – a Git tool but not SQL, a debugging skill but not frontend, recent preferences but not old project notes. Greyed-out nodes are skipped entirely. This deferred loading is why agents can have hundreds of tools and skills without running out of context.

How an agent works

Two triangles contrast the workflow: the old way (you type, paste, import, export, compare, run; chatbots only emit text and figures) versus the new way (you say, agents do the work, you review). Both start static; one click lights the matching node on both at the same instant and cycles, to make both the contrast and the loop explicit. There are no commands to memorize, just keep talking. (Static in print and under prefers-reduced-motion.)

Say It, Done

Say it, done: literature review

You say:

Review the literature on Shanghai’s lane (lilong) neighborhoods.

What the agent does:

1. Read the folder of PDFs
2. Identify each study’s research design
3. Extract key findings and data sources
4. Sort by theme and year
5. Build a literature matrix
6. Draft the review

→

Output:

• Literature matrix (Excel)
• Reference list
• Key findings per paper (detailed Word file)
• Summary by category (short Word file)
• First draft of the review

Time: < 5 minutes

Same principle as the data example: you described the task in natural language, the agent read your actual PDF folder. No manual reading required to build the matrix – that part is logistics. (Lilong are Shanghai’s traditional alleyway neighborhoods, a classic topic in urban sociology.)

Say it, done: data analysis

You say:

Report missing data by country and year in the World Bank indicators.

What the agent does:

1. Read your data file
2. Identify the country and year variables
3. Compute missing rates for each group
4. Write the code and make charts
5. Produce summary tables and figures
6. Assemble them in Quarto
7. Export to Word, PDF, and HTML

→

Output:

• Code
• Figure files
• Word document
• PDF
• HTML page

Time: < 10 minutes

The key here: you described the task in plain language, naturally. The agent read the actual files in your project – no copy-paste, no explaining your data structure. It figured that out itself.

Say it, done: your turn

Pick a research task and click “Let the agent try” to see how it plans. (Simulated locally; no internet needed.)

Task

Thinking…

A participant-driven companion to the previous slide. The plan is pre-authored per task (deterministic, offline). It is a simulation of the agent’s planning, not a live LLM call, so it runs without network. The time estimate scales with data size and complexity.

Say it, done: conference talk

You say:

Make a 15-slide deck.

What the agent does:

1. Read the paper draft for its core argument and structure
2. Read the literature matrix and abstracts
3. Read the figures and regression results
4. Design the slide structure and narrative arc
5. Write the slides in Typst
6. Compile to PDF

→

Output:

• Typst source (still editable)
• Compiled PDF slides
• Speaker notes for every slide
• A first-draft script

Time: < 15 minutes

Third example: synthesis task. The agent reads across all project artifacts – literature matrix, regression tables, figures – and produces a coherent slide deck in Typst. This is the hardest task of the three: it requires judgment about narrative structure, not just data processing. But the agent can do a credible first draft in minutes.

Say it, done: polishing a paper

You say: Polish this paper to ASR standards.

5 reviewers in parallel: style · structure · grammar · citations · figures

Output: revised paper + an edit log per round
Time: < 30 minutes

Multi-agent orchestration: reviewers run in parallel, not sequentially, and the orchestrator decides when the paper is “good enough” to stop looping. This mirrors how a senior researcher would coordinate a team of specialized RAs – except the whole team works simultaneously and never gets tired.

Acceleration and empowerment

Prof. Chen writes a CSR paper in three days

Scenario 1: acceleration. An experienced scholar + an agent = an empirical CSR paper on community governance in Shanghai, in three days.

What Prof. Chen brings

• Rising-star faculty at Shanghai University
• Years of community research
• Data and fieldwork materials
• Fluent with agents
• Three days

The agent’s three roles

• Day 1: data analyst
• Day 2: literature assistant
• Day 3: paper writer

CSR = Chinese Sociological Review, an English-language SSCI journal. The scenario is illustrative: a seasoned scholar with data and domain expertise uses the agent to compress execution time, not to replace the thinking.

Three days, three jobs

① Day 1 · Data analysis

The agent plays data analyst.

• Clean the data ← fix errors, fill gaps
• Analyze ← descriptives, regressions
• Report ← tables, figures, interpretation

② Day 2 · Literature review

The agent plays literature assistant.

• Search the literature ← titles and DOIs
• Fetch PDFs in bulk ← via library access / Zotero
• Synthesize ← summaries, review draft

③ Day 3 · First draft

The agent plays paper writer.

• Core arguments ← distilled from the review
• Core results ← distilled from the analysis
• Full draft ← in CSR structure

Prof. Chen reviews the output and steers the direction (the harness); the agent does all the execution.

A high schooler writes an ASR introduction

Scenario 2: empowerment. A tech-savvy science student who isn’t a native English speaker can still write an ASR-quality introduction.

In principle, the rhythm, tone, style, and voice of academic writing can all be learned from published papers.

The traditional path

Grad school → years of reading → find your own style → competence, eventually

The agent path

Download papers → agent learns the style → generate candidates → synthesize the best version

The key provocation: writing style – the rhythm, argumentation pattern, sentence structure – is a learnable, transferable pattern. An agent can extract it from hundreds of papers and apply it to a new topic. This is not about replacing original research; it is about democratizing the ability to write in a way that passes the style test of top journals.

Three phases: learn · execute · teach

① Learn

The human sets the direction; the agent builds the knowledge base.

• Build a corpus ← 100+ recent ASR papers
• Pick a topic ← e.g. “A century of educational inequality in the U.S.”
• Model the style ← agent extracts rhythm and argument logic

② Execute

Generate in bulk · filter · synthesize the best introduction.

• Generate ← 100 introductions, one per template paper
• Review ← another agent picks the 20 strongest
• Synthesize ← a final agent condenses them into one

③ Teach

The AI becomes a tutor: it explains why it’s written this way.

• Line by line ← each paragraph’s strategy, in the student’s own language
• Language ← key academic terms and phrasings
• Next steps ← a personalized reading list

The result: a student with no sociology training can see what makes this introduction good.

Three phases compressed onto one slide: learn (build the corpus and style model), execute (generate → filter → synthesize), teach (explain each paragraph, teach the English, chart the next step). The agent is not just a writer – it is a tutor.

Two ideal types

Dimension	Acceleration	Empowerment
User	Advisor (expert)	Student (novice)
Agent’s role	A student who does the work	An advisor who does the work
Core mechanism	Delegating repetitive work	Learn, execute, teach
Human contribution	Direction + quality control	Direction
Agent contribution	A tireless workhorse	A stroke of genius
Best-case output	A paper in three days	An ASR introduction

Advisors save time; students gain knowledge.
Perhaps one day the roles of advisor and student will both fade.

Capabilities and limits

What are agents good at?

Area	Rating	Concrete tasks
Computation and code	★★★★★	Organizing files, cleaning data, building variables, running regressions, replication, figures
Structured knowledge	★★★★★	Literature screening, organizing materials, codebooks, abstracts, findings, drafting methods and appendices
Research design and framing	★★★★★	Can help sharpen questions and propose structure and identification strategies, but the researcher must sign off
Theory and contribution	★★★★★	Can help summarize possible contributions, position the work, and spot logical gaps, but cannot replace expert judgment
Surveys, interviews, fieldwork	★★★★★	Researchers on site, building rapport

This table helps skeptics locate exactly where agents help without overpromising. The “contribution judgment” row deliberately scores low – this builds credibility for the rest of the talk.

Your personal Wikipedia

Let the agent keep improving your knowledge base; it gets smarter the more you use it.
See: karpathy/llm-wiki

The architectural contrast is the core message. Left: RAG scatters the query across raw documents on every call – fragments retrieved, temporarily reassembled by the LLM, then discarded. Nothing persists. Right: the Wiki inverts this. Raw sources stay immutable at the bottom. Above them grows a living wiki – LLM-maintained markdown pages with cross-references – built through Ingest, health-checked by Lint, queried with full citations. The key enabler: maintenance cost that defeats every human knowledge system drops to near-zero when the LLM absorbs it.

Four quadrants of research

Direct framing, no softening. The competitive dynamics are real. The point isn’t to scare people – it’s to clarify that “not using AI” has costs, not just benefits. The upper-right quadrant is where this talk is pointing.

Four kinds of capital for the future

CORE

① Social interaction

Political capital + social capital

CORE

② Data and materials

Material capital

MUST-HAVE

③ The disciplinary frontier

Cultural capital

MUST-HAVE

④ Steering agents

Know-how

A mirror from history

From the “car industry” to the “car society”: urban space, marriage and family, car culture, geopolitics. (Lü Peng, June 8, 2026; in Chinese)

This is the “what you need” complement to the quadrant slide. Items 1 (social interaction / political & social capital) and 2 (data resources / material capital) are the irreplaceable core – no AI can substitute for relationships and domain data access. Items 3 (disciplinary frontier / cultural capital) and 4 (agent skills / technical capital) form a pair: you must hold both. Lose frontier knowledge and your AI output is shallow; lose agent skills and you can’t keep pace. Together they keep you competitive.

A mirror from history (why ① social interaction is core): the car society. When the car went from luxury to mass consumer good, it reshaped not just transportation but urban space (suburbanization), family structure (commuter marriages), consumer culture (drive-ins, road trips), and even geopolitics (oil dependence). Scholars realized only later that we had planned for a “car industry” and underestimated the complexity of a “car society”. Congestion, pollution, traffic deaths, and energy dependence were not problems car engineers could solve; they took urban planners, public-health experts, legislators, and community organizers. Today’s frontier industries will penetrate society even more deeply than the car – they reach directly into our bodies, cognition, and social lives. Silicon Valley vs. Route 128 – Saxenian’s Regional Advantage: Route 128 was firm-centric, hierarchical, secretive, and hostile to job-hopping; Silicon Valley rested on an open, networked social structure – engineers kept socializing after work, venture capitalists were former founders, talent moved freely, and failure was tolerated rather than stigmatized. You can copy Silicon Valley’s office parks and term sheets, but not an industrial system built on regional social networks. When governments race to build “future industry” zones and measure them almost entirely by output, patents, and funding raised, that is itself a systematic neglect of the social dimension.

AI exposure ≠ job loss

Embodied craft and on-site work are key skills for the new era. (ShanghaiTech CEISD et al., July 21, 2026; in Chinese)

Chinese empirical counterpart to the four-capitals slide. ShanghaiTech CEISD built a dynamic Chinese C-O*NET (65,200 tasks, 1,267 occupations) from 700M+ job postings across five platforms, and scored every occupation on 13 task dimensions. The “horse race” of correlations with hiring-share change: on the shrinking side, LLM exposure is the strongest predictor (-0.47); on the expanding side, embodied craft (+0.55) and on-site dependence (+0.53). High exposure alone does not mean high risk – Java developers and call-center agents score similarly on exposure, but the developer’s architecture / incident-response / cross-system judgment keeps demand and wages rising while call-center hiring shrinks. The report’s headline: not occupational extinction but task recomposition – 21M jobs (18.6% of postings) were created by task reorganization, and the pressure concentrates on entry-level (1-3 yr, 5k-8k RMB monthly salary) positions. Tie back: this is exactly why ① social interaction and ② data and materials are the core capitals – the resilient tasks are the situated, embodied, high-responsibility ones. For an international audience, note that the pattern echoes U.S. findings on AI exposure (e.g. Eloundou et al. 2024) – exposure measures tasks, not jobs.

Skills on the move

From handwriting to brain–computer interfaces

Handwriting

Cursive is fading

→

Typing

Keyboard / autocorrect

→

Voice

Dictation

→

Brain–computer interface

Reading neural signals

One task, four ways to input it.
Are these skills becoming obsolete?

Four ways to get what is in a head onto a page, arriving in sequence: handwriting, typing, dictation, and now neural input. The natural question is “are the earlier skills being deprecated?” The thesis of this section: “deprecated” is the wrong frame. Technology rarely deletes a skill; it relocates it. The useful question is “where does it go, what hidden by-product goes with it, and who decides?” Handwriting is the perfect local hook: many U.S. schools dropped cursive, and Chinese speakers now joke about “character amnesia” (提笔忘字) – knowing a character on screen but being unable to write it by hand.

Why do we still watch humans play chess?

Chess engines left humans behind long ago (Deep Blue in 1997; AlphaZero and Stockfish dominating since 2017). Yet since 2020, the number of people watching and playing human chess has exploded.

If machines are so much better, why do we still watch people play?

We watch for the uncertainty

The thrill of sport is that nobody knows the outcome. No one wants to watch a guaranteed flawless win.

We identify with people who can lose

People blunder, choke, and show courage. A machine can’t lose, so there’s nothing to identify with.

Sport, art, and competition are stages where we watch how people stay human under pressure.
Difficulty and mistakes aren’t flaws; they’re how our humanity shows.

Chess engines have been superhuman since Deep Blue (1997) and decisively so since AlphaZero/Stockfish (2017 onward), yet human chess viewership and play boomed from 2020 to 2024. Engine versus engine is a benchmark, not a sport. The mechanism is two-fold: the uncertainty-of-outcome hypothesis in sports economics, and we identify with the fallible because empathy needs the possibility of failure. The deeper point: sport and art are arenas where we watch what it is to be a human under pressure. Imperfection is not a bug in the spectacle; it is how humanity becomes visible.

Technology doesn’t delete skills; it moves them

Skill shift	What happens to people	What it means with AI	Examples
Fading	Drops out of daily use, or becomes a hobby	Largely replaced by AI	Abacus arithmetic, slide rules, penmanship, wilderness survival
Coexisting	Both persist, each with its own role	AI creates new forms; the live human still has value	Track and field, dance, live concerts, home cooking
Upgraded (accelerated / empowered)	Experts get faster, novices get stronger	The bar drops, execution is outsourced, humans steer	Translation, Go, photography, design, coding, research
Newly born	New technology creates new demand	AI creates new jobs and occupations	Drivers, typists, AI trainers

The core reframe. Distinguish task vs. skill vs. occupation: technology automates tasks; skills and occupations recompose. When the routine core is automated, the human residue lands in one of five fates. Moravec’s paradox explains the counterintuitive order; Jevons paradox warns that making something cheaper often increases total demand. Note that the “integration” fate can be a transition, not a destination. The “newly born” row is historical on purpose: cars created drivers, typewriters created typists.

The combine harvester

The harvester story

Heilongjiang, northeast China, 1956. You’re cutting wheat with your village’s farming team. One sickle, about a sixth of an acre a day.

One day, the state farm next door brings in a machine: a Soviet-built “Stalinets-6” combine harvester. It roars across the field and easily harvests 65 acres in a single day.

The villagers gather round and stare for a long time.
Someone says, “That thing costs a fortune. We could never afford it.”
Someone else says, “A machine’s not as steady as a hand. If it ruins the crop, who’s to blame?”
And some just stay silent, gripping their sickles a little tighter.

Harvester = AI agent　　Sickle = traditional methods　　Grain = your research

A true-to-period vignette: in the mid-1950s, Soviet-built Stalinets-6 combines arrived at state farms in Heilongjiang, China’s northern grain frontier, while neighboring villages still harvested by hand. The same scene played out across the American Midwest a generation earlier, when horse-drawn binders gave way to combines; mention that if the room needs a closer reference. Let the three reactions land – they set up the next slide.

Five kinds of resistance

① “I don’t need a harvester”

AI won’t touch my field.

But every discipline has grunt work.

② “Harvesters are too expensive”

The sickle is cheap and good enough.

But time is the biggest cost of all.

③ “I’ll never learn to drive one”

New tech feels daunting; the sickle feels safer.

But the learning curve is flattening fast.

④ “Who’s to blame if it ruins the crop?”

Misused AI: hallucinations, leaks, logic gaps.

Fair, but learn to drive; don’t smash the machine.

⑤ “The big farms will put us out of work”

Fear that AI deepens inequality.

Fair, but not using it ≠ not being affected.

The farming metaphor works across cultures: everyone recognizes the moment a machine shows up in a field worked by hand. Each resistance is legitimate but insufficient as a reason to disengage entirely. The fourth fear is the most sophisticated: AI can actively produce harm – hallucinations, data leaks, flawed reasoning that looks plausible. Unlike the others (which are about missing out), this one is about active damage. The rebuttal: the risk is real, but the answer is learning to operate the machine safely, not refusing to engage. Not using the tool doesn’t protect you from its effects – others will use it.

Three kinds of overoptimism

Owning a harvester doesn’t guarantee a harvest:

①

First, you need a field

You need a research question and data.

②

You need to know the seasons

Research judgment can’t be outsourced.
When to explore and when to converge is your call.

③

Every field is different

Your data, methods, and writing style are your own.
Tailor the agent’s setup to your research.

This slide tempers the enthusiasm. The three points map to: domain knowledge, research judgment, and personalization. Without these, AI produces generic or misguided output. The agent is a multiplier – but a multiplier on zero is still zero.

AutoResearch: five levels of autonomy in research

Source: Tie et al., working paper · arXiv:2605.23204

Getting started

Using an agent

Level 1: system setup

Install the tools and prepare your environment.

• Install Git (check “Add to PATH”)
• Install VS Code (check “Add to PATH”)
• Install the Claude Code or Codex extension in VS Code

One-time job. About 20 minutes.

Level 2: task setup

Tell the agent how to think about your research:

• How to frame research questions
• How to handle literature reviews
• How to construct variables
• Which methods to use
• How to format tables and figures
• Your writing style

A one-time investment that pays off in every project.

The two-level framing is important. Level 1 is the technical barrier that most people think about. Level 2 is the intellectual configuration – and that’s where the real compounding value is. Once you’ve taught the agent how your group does sociology, it carries that into every project.

Compounding returns: set up once, benefit for years

The core principle:

Any repetitive task only needs explaining once.
Any similar setup only needs designing once.

Example 1

Set up “how to handle PSID data” once, and every new project inherits it.

Example 2

Write “model literature reviews on [a paper]” once, and the agent follows it every time.

The first project is the hardest to set up; every one after is easier.
That’s compounding, not linear growth.

The compounding return is the key economic argument. The first project is where you invest – you teach the agent your group’s conventions, your preferred methods, your writing style. After that, the agent carries all of that forward.

A sample project configuration

The core of a CLAUDE.md configuration file:

# Research project setup

## Data processing rules
- Always use Stata for data cleaning
- Variable construction: follow the coding scheme in @Zhang-Miao-2025.pdf

## Output format
- Regression tables: use esttab
- Figures: white background, 7×5 inches
- Citations: American Sociological Review style

## Project structure
data/raw/     ← raw data (read-only)
data/clean/   ← cleaned data
do-file/      ← analysis scripts
output/       ← figures and results

This is a real example of what goes in CLAUDE.md. Notice: it encodes your group’s methodological conventions, not just file paths. The agent reads this at the start of every session. You write it once and it travels with the project.

Break big tasks into small ones

When the AI’s work disappoints, it usually means the task was too big and needs splitting.

When AI gives bad answers, the problem is usually that you asked too big a question. Decompose the research task into subtasks small enough that the AI can handle each one reliably, verify each piece, then combine them. The hourglass shape – one big task splits into many small ones, then converges into one result – is the universal pattern for productive agent use.

Steering the agent

The road has curves. Floor it in a straight line and you’ll crash.

Three approaches to using AI agents: (1) No agents – you do everything manually. Safe but painfully slow. (2) Full automation – fire off a huge prompt and let it run unsupervised; it goes off the rails at the first complex turn. (3) Strategic driving – let the agent handle straight stretches, then take the wheel at turning points. Delegate the boring parts, intervene at the critical moments. The road has turns: if you don’t steer, you crash; if you never drive, you never arrive.

A crisis for social science

The stronger the tools, the harder we must guard the researcher’s judgment:

Contamination at the source

As Nature reported (2026):

• Up to 45% of survey responses may be LLM-generated

Stronger individuals, narrower science

A Nature study also found:

• Collective topic diversity fell 4.6%
• Attention concentrates on a few “star problems”

Don’t floor it in a straight line: pick a destination, turn on navigation, and drive safely.

Hao et al. (2026). Nature. · Adam, D. (2026, June 2). Nature.

The honest counterweight, kept light and framed as supporting the “strategic driving” message. The -4.6% collective narrowing comes from the same Hao et al. study cited on the evidence slide (a clean single source). The 45% contamination figure is from a Nature news feature. The point is not “AI is dangerous” but “this is exactly why the human must stay at the wheel”: it reinforces, rather than rebalances, the talk’s optimistic arc.

How an agent runs

Key architectural insight: ALL execution happens locally. File reading, file writing, running shell commands, permission checks – your machine does all of that. The cloud does exactly ONE thing: language model inference. Privacy implication: code, data file contents, and command outputs are all sent to the API.

An agent’s native language: plain text

Agents can only read and write text, so file format determines compatibility.

Works natively ✓

The agent reads and writes these directly:

• .do / .R / .py – analysis scripts
• .md / .qmd / .tex – writing and slides
• .csv / .json – data
• .bib – references
• Git repositories, config files

Needs a bridge ⚠

Other formats must be bridged, parsed, or converted first:

• Word .docx
• PowerPoint .pptx
• Excel .xlsx
• PDF
• .dta / .rds and other binary data

The terminal (and therefore the agent) operates entirely in plain text. Text files are what agents read and write natively. Binary formats – Word, PowerPoint, Excel – are opaque blobs that agents cannot directly parse or edit. The reassurance for this audience: Stata .do files are actually plain text.

Working with Word documents

Original Word → Markdown → agent edits → Markdown → Word, but the loop doesn’t close.

✓ Kept

Body text, heading levels, tables, citations, lists

✗ Lost

Track changes, comments, custom styles, fonts, headers and footers

The rule: work in plain text and convert at the end. Accept some drift; save your attention for content.

The round-trip Word → Markdown → Agent → Markdown → Word does NOT preserve everything. What survives is the content layer; what gets lost is the presentation and collaboration layer – track changes, comments, custom styles. python-docx can mitigate some loss but cannot fully recover tracked changes. The principle: work in text, convert at the end.

A cleaner route: edit the XML directly

A .docx is a ZIP of XML. The docx skill edits that XML directly, so the loop closes.

Item	Markdown route	XML route
Content (text, tables, headings, citations)	✗ Lost	✓ Kept
Formatting (styles, fonts, headers and footers)	✗ Lost	✓ Kept
Track changes and comments	✗ Lost	✓ Kept

A .docx file is a ZIP container of XML. The docx skill provides unpack.py and pack.py; between them the agent edits XML directly, wraps changes in tracked-change markup, and preserves every original style, font, image, and embedded element. The result: format fidelity AND revisions appear as tracked changes. Trade-off: slower and requires OOXML knowledge; for bulk rewrites the Markdown route is faster, for precision editing with format preservation XML direct-edit is right.

Choosing a tool

Which tool should you pick?

	Claude Cowork	Claude Code	Codex	Gemini CLI
Provider	Anthropic	Anthropic	OpenAI	Google
Core model	Claude	Claude (Opus / Sonnet)	GPT / Codex	Gemini
Interface	Desktop app	Terminal + IDE + desktop	Terminal + IDE + app	Terminal + IDE
Entry price	Claude Pro ~$20/mo	Claude Pro ~$20/mo	Limited free; Plus ~$20/mo	Limited free; AI Pro ~$20/mo
Best at	Office files, no coding	Research projects, large codebases	All platforms, generous Plus limits	Long context, Google ecosystem

All four work the same way: you ask in plain language, so switching tools doesn’t cost you the skills you’ve learned.

Comparison of the four-tool lineup. All four follow the same natural-language pattern, so skills transfer across them. Claude Cowork is Anthropic’s desktop “work agent” for knowledge workers: it operates on local folders and Office files with no coding. Claude Code is the terminal/IDE coding agent this talk demos, strong on multi-file research projects. Codex (OpenAI) is the terminal + IDE + app agent in the ChatGPT ecosystem; Plus limits are generous and a limited free tier exists. Gemini CLI (Google) offers a very long context window and ties into Google Workspace; its free quota was reportedly cut in 2026, so check current limits. Prices are approximate as of October 2026 – verify on the official pages before the talk. Cursor is deliberately left out: after the 2026 acquisition its pricing was restructured around Grok, and it is in flux.

Work agents vs. code agents

Same core, two forms. Both run code, handle local files, search the web, and carry out multi-step tasks.

Code agents · programming depth

• Whole-project view: knows structure and dependencies
• Direct edits: changes files, runs the shell and tests
• Parallel subagents: explore / plan / code
• Hooks and skills: automated checks, reusable workflows
• MCP: connects external tools and data

Work agents · office automation

• Office files: PowerPoint / Excel / Word
• Scheduled tasks: daily briefs, recurring jobs
• Browser use: navigates sites like a person
• Connectors: email, calendar, cloud drives
• Low barrier: built for everyday office work

For coding and research, use a code agent; for documents and reports, use a work agent.

Siblings, not competitors – Claude Cowork runs on the same agent engine as Claude Code, so the overlap is real: both run Python, touch local files, search the web, and execute multi-step tasks. The split is home turf. Claude Code lives in the terminal and IDE: repo-level understanding, direct file edits and test runs, parallel subagents, hooks, skills, and MCP servers. Cowork lives on the desktop for knowledge workers: it works inside a folder you grant it, produces finished PowerPoint/Excel/Word files, runs scheduled tasks, and uses connectors and a browser. Rule of thumb: if the deliverable is code or analysis you iterate on, choose a code agent; if it is a recurring office workflow, choose a work agent.

Which one fits you?

Answer the first two questions to get a live recommendation and a runner-up.

1. Main task?

2. Budget?

3. Preferred interface? (optional)

Answer the questions above to see which tool fits you best.

Interactive recommender. Question 1 (task) and question 2 (budget) are required; interface is optional. Scoring is a simple weighted sum over the four tools. Prices shown are approximate (October 2026); remind the room to check current pricing.

Questions and discussion

The AI is fast. You are the one who knows what matters.

Thank you!

Minghong Shen
mhshen@sas.upenn.edu

Slides:
https://www.socimh.com/slides.html
Hands-on guide (in Chinese):
socimh.com/cc-guide/

Leave ample time for questions. Common objections: data privacy (configure local models or an API with appropriate data agreements); reliability (always review its work – it is a tool, not an oracle). Have TUTORIAL.md ready to show on screen.

Appendix

The limits of agents

Knowledge cutoff

It can’t see the latest research: newly published papers and recent developments may be unknown to it.

Fix: you supply the latest literature; the agent integrates it.

Context window limits

Limited context means it can’t hold a very large knowledge structure in view, or process a huge dataset in one pass.

Fix: work in chunks; keep your setup modular.

Agents don’t replace professional expertise. They are your execution layer; you remain the owner of the research.

Stating limitations explicitly builds trust. The audience will think of objections – preempt them. The point is not “agents are perfect” but “the things they do well are exactly the things that eat your time.”

When a famous advisor becomes a skill

Zhang Xuefeng’s “cognitive operating system”

Zhang Xuefeng: China’s best-known and most controversial college-admissions advisor, followed by tens of millions.

Five mental models

• Choice beats effort
• Society is a sieve
• Work backward from the job market
• Class realism
• Controversy is reach

Data foundation

• 5 books
• 15+ major media interviews
• 30+ first-hand quotes
• 8 decision heuristics
• A full life timeline

Zhang Xuefeng – China’s most controversial college-admissions advisor – encoded as a Claude Code skill. Not a chatbot, not a quote database: a runnable cognitive system. Five mental models, eight decision heuristics, built from books and interviews. The point is not Zhang Xuefeng himself but the technology: an expert’s reasoning framework, now invocable by anyone. For a Western analogue, imagine the same treatment for a famous admissions consultant or a well-known investor’s shareholder letters.

What will agents mean for white-collar work?

It depends on:

1. How much of the work in this occupation can AI accelerate or empower?

2. Can the occupation’s scope expand, or is it stuck in zero-sum competition?

3. How high are the barriers to entry?

Situate your own field. Sociology: lots of lit review, data cleaning, methods work – all high-replaceability. Willingness to adopt is mixed. The combination creates a real window for early movers.

Is AI narrowing science?

A Nature bibliometric study used an ML classifier on 41M+ papers (1980–2025, six fields) and flagged 310,000 that clearly used AI. The ledger cuts both ways:

The individual ledger: a win

• Scientists who use AI: 3× the papers, nearly 5× the citations
• Junior scientists leave academia less often and become PIs sooner

The collective ledger: a loss

• Topic coverage 4.6% narrower
• Papers building on each other down 22%, all orbiting AlphaFold-style “star studies”

“We’re like pack animals.” (Evans) Using AI is a personal moat; but when everyone climbs the same mountain, vast territory goes unexplored.

Evans & Xu team (Nature) · Science news coverage: Zhao, C. (2026-01-22). Science 391(6783). DOI: 10.1126/science.aef6710

The full picture behind the -4.6% figure already cited on the “crisis for social science” slide – same underlying Nature study. Mechanism is a self-reinforcing herding loop: hot problems accumulate data → AI tools work best where data is abundant → AI adopters there publish and get cited more → more scientists pile in → topics narrow further. Evans: “We’re like pack animals.” Yale anthropologist Lisa Messeri (external commentator, not an author) calls it a loud alarm: a tool that benefits individuals while eroding science deserves serious reflection. Caveats worth stating if asked: (1) correlation vs causation – stronger scientists may simply adopt AI earlier; (2) 4.6% is not a huge absolute narrowing, Messeri’s wording is heavier than the data; (3) topic-coverage definition is not disclosed in the News piece; (4) figures here are Zhao’s journalistic account of the Nature paper. Constructive counterpoint: teams are already using LLMs + concept maps to suggest cross-field topics – using AI to counteract AI-induced narrowing.

The evidence: more than a hunch

Large-scale studies point the same way

Individuals · adopters pull ahead

Researchers who use AI

• About 3× the publications
• About 4.8× the citations
• Full-population evidence from 41.3M papers

The same study also finds collective topic diversity down 4.6%

Who gains most · novices

Generative AI for knowledge work

• +14% productivity on average
• +30–35% for novices and less-skilled workers
• A field experiment with 5,172 support agents

The less skilled you are, the bigger the gain: hard evidence for “empowerment”

Hao et al. (2026). Nature. · Brynjolfsson, Li & Raymond (2025). Quarterly Journal of Economics.

Independent peer-reviewed evidence backing the 180/60/20 curve. Hao et al. (Nature 2026): across 41.3M papers, AI-using scientists publish 3.02× more and receive 4.84× more citations. The same study finds collective topic diversity shrinks 4.63%; we pick that up on the flip-side slide. Brynjolfsson et al. (QJE 2025): a randomized field experiment on 5,172 customer-support agents showing +14% average productivity, with the least-skilled workers gaining 30–35%. This is direct empirical support for the deck’s “empowerment” thesis that agents compress the skill ladder most for novices.