The short version
- The question. More than 81 percent of Germans perceive society as divided. We wanted to know what people write when nobody asks, and we are counting around 2.5 million public YouTube comments.
- Cheap before expensive. The analysis runs in four stages. Counting and rules see every comment. The language model sees a sample of 52,433, which is about two percent.
- Check first, interpret later. A language model assigned 49 recurring claims. Checked by hand, about eight in ten assignments were right. Rankings hold, small differences do not.
- Statements, not people. Names are never stored, accounts exist only as pseudonyms, and only totals are shown.
Contents10 sections
In a 2025 survey by the Mercator Forum Migration and Democracy, more than 81 percent of Germans perceive society as divided. The sociologists Steffen Mau, Thomas Lux and Linus Westheuser write in "Triggerpunkte", citing a series of studies, that over time there is, for Germany, hardly any evidence of a fundamental ideological polarization.
Both are true at once. Four in five feel a rift, and in people's attitudes it is hard to find. That was the reason for a project of our own. It is called "Wie gespalten sind wir wirklich?", how divided are we really. This piece shows why we do it, what we built and how you analyze something like this.
1 of 5
Own count, as of November 17, 2026
about 2.5 m
public comments under 21 German YouTube channels. Each point stands for about 800.
First they lie in a heap. Then all of them are counted and sorted by rules, without a model.
Stage 3, a cheap language model
52,433
comments are read by the language model. That is about two percent of the stock.
Two draws: a random one and one stratified by channel and month.
Codebook, version 2
49
recurring claims are in the codebook.
Eleven of them appear about equally often under channels on the left and on the right.
40 assignments checked by hand
about 8 in 10
assignments made by the language model were right.
The bright points are the errors. Rankings hold, small differences do not.
834,369 comments from 288,725 accounts
19%
of all comments are written by the most active one percent of accounts.
Whoever reads a comment section hears volume first.
- about 2.5 mOwn count, as of November 17, 2026. public comments under 21 German YouTube channels. Each point stands for about 800. First they lie in a heap. Then all of them are counted and sorted by rules, without a model.
- 52,433Stage 3, a cheap language model. comments are read by the language model. That is about two percent of the stock. Two draws: a random one and one stratified by channel and month.
- 49Codebook, version 2. recurring claims are in the codebook. Eleven of them appear about equally often under channels on the left and on the right.
- about 8 in 1040 assignments checked by hand. assignments made by the language model were right. The bright points are the errors. Rankings hold, small differences do not.
- 19%834,369 comments from 288,725 accounts. of all comments are written by the most active one percent of accounts. Whoever reads a comment section hears volume first.
Why an analysis like this
Surveys measure what people say about themselves when asked. Comments are the counterpart. Nobody asked, nobody offered answers to choose from, and there are a great many of them. If you want to know what people argue about and what the camps agree on, you find material there that a survey does not deliver.
For companies this is not an academic question. Customers, applicants and employees are the same people who write there. If you hear only the volume, you take a loud minority for the majority.
And there is a second reason. With Balane I build software for companies and advise them. The same technique that sorts comments here also sorts customer feedback, support requests or open-ended survey answers. A public example shows how to do it cleanly and where it fails.
What we did
The basis is around 2.5 million public comments under 21 German YouTube channels. We provisionally classified six channels as left and six as right; four have no recognizable leaning, five are non-political. The non-political ones are the control: they show what about a comment section is simply a comment section.
The data comes through the official interface, the YouTube Data API. No scraping. The API bills in units, and that is where it is decided whether a project like this stays affordable: one page of a channel's upload list costs one unit, one search costs a hundred. We never use search, and we keep to our own daily limit of 9,000 units.
Storage follows three rules:
- Append only. Raw data stays as it arrived. A new fetch is a new record with its own timestamp.
- No duplicates. That the same comment is not counted twice is guaranteed by keys in the database, not by program code.
- Every number has an origin. Every value points to the fetch it came from, and every fetch to the program run with its time and code version.
In the test run with 79 sample videos, 100 percent of the reported comments were stored, without duplicates. Only then did we widen the collection.
Pseudonymization: statements, not people
We study statements. No political profiles of individual commenters are created, neither as a table nor as a label. Technically that means:
- An account's identifier is turned into a pseudonym with a secret key before it is stored (HMAC). The plain value is never stored.
- Display names are stored nowhere.
- Mentions of other accounts in the comment text are replaced before storing.
- A language model receives plain text, never an identifier or a pseudonym.
- All analyses show totals. No single account is classified politically.
The pseudonym has exactly one purpose: counting how many accounts write and whether the same accounts turn up under several channels.
Text analysis in four stages
The most important decision was the order. Each stage gets only what the previous one could not settle.
- Counting and rules, without a model. Database queries, the language tool spaCy, word lists. How many comments are replies to others? How many contain an exclamation mark, capital letters, emojis? This runs over every comment and costs next to nothing.
- Local open models. For everything rules cannot grasp but that needs no expensive model.
- A cheap language model. Only for a sample and for unclear cases, with a hard cost limit.
- Statistics. Measured against comments that people classified by hand.
The first stage already delivers findings. In capital letters and emojis, political and non-political channels differ clearly. Left and right, by contrast, mostly lie close together. Under non-political channels, short comments are more common.
It also shows what a simple sentiment analysis cannot do. We count negative and positive words using SentiWS, a sentiment dictionary from Leipzig University. That is crude. A dictionary recognizes neither irony nor negation. For large volumes it is a pointer; about a single comment it says nothing. Whether a comment is hostile is therefore something we do not measure yet.
The codebook: content analysis with a language model
For the content, counting is not enough. Here the analysis becomes a quantitative content analysis as social research knows it: you define categories, write them into a codebook and assign every text to a category. The only new thing is who does the assigning.
Our sample comprises 52,433 comments. It consists of two draws: a purely random one and one stratified by channel and month, so that small channels and quiet months do not disappear. In its second version the codebook contains 49 recurring claims. A language model assigns to each comment which of them it contains.
Eleven of the 49 claims appear about equally often under channels on the left and on the right. The most common shared one is: the media report one-sidedly and leave out what matters. Under left-leaning channels it appears in 1.9 percent of comments, under right-leaning ones in 2.4 percent. These are the commenters' claims. We have not checked them, and they are not our opinion.
Is it right? Inter-rater reliability with a machine
A language model always assigns something. It does not say whether it is right. In content analysis the question is called inter-rater or intercoder reliability: do two coders reach the same result on the same text? In our case one of the two is a machine.
We checked 40 assignments by hand. About eight in ten were right. That is a small check, and we treat it as one. A simple rule follows for everything we show: rankings hold, small differences do not. That one claim is by far the most common survives an error rate of two in ten. That it reaches 0.5 percent on the left and 0.7 on the right is a hint, not a finding.
The next step is bigger. 1,000 comments are being classified by hand. Every model is tested against them before it may assign more. Every feature also carries its procedure and version. If a procedure changes, the version goes up and the old results stay.
First numbers
The collection is still running. On November 17, 2026, the stock stood at around 2.5 million comments. The numbers in the table are from November 13 and 14, when it was 1.3 million.
| Number | What it says |
|---|---|
| 834,369 comments, 288,725 accounts | The base under long videos of the 21 channels |
| 59% | of accounts wrote exactly once |
| 19% | of all comments are written by the most active one percent of accounts |
| 82% | of accounts write under one single channel only |
| 11 of 49 | claims appear about equally often on the left and the right |
| 19 of 21 | channels contain the claim that the media report one-sidedly |
So whoever reads a comment section hears volume first. A few accounts write a lot, and that is similar in every group.
What to take from this for your own project
The technique is not tied to YouTube. Anyone who wants to analyze customer feedback, reviews or open-ended answers faces the same decisions.
- First the question, then the data. The selection of channels was fixed before the first comment was loaded. Otherwise the result is already in the selection.
- Use the official interface and do the math. What a call costs decides the architecture.
- Store raw, never overwrite. Everything derived must be reproducible.
- Cheap before expensive. What can be counted is counted. The language model gets the rest, and a sample of that.
- Measure the model against people. Before the first interpretation, not after.
- Write down the limits. Every number comes with its base, its date and its uncertainty.
Count what can be counted. Check what a model claims.
What the analysis cannot do
The groups consist of four to six deliberately chosen channels. The results hold for these channels, not for YouTube and not for the Germans. Commenters are the loud part of an audience. The classification of the channels is provisional and ours. And everything here is an interim result.
Sources
- Mercator Forum Migration and Democracy (MIDEM): Polarisierungsstudie 2025, press release (German). forum-midem.de
- Steffen Mau, Thomas Lux, Linus Westheuser: "Triggerpunkte. Konsens und Konflikt in der Gegenwartsgesellschaft", excerpt at the Federal Agency for Civic Education (German). bpb.de
- SentiWS, sentiment dictionary of Leipzig University, version 2.0, CC BY-NC-SA 3.0. wortschatz.uni-leipzig.de
- YouTube Data API: cost per call in quota units. developers.google.com
Our own numbers. Project "Wie gespalten sind wir wirklich?", data as of November 13 and 14, 2026. Sample for the claims: 52,433 comments, procedure codebook version 2.