> Software developers in the US are seriously expensive, using them for data labeling would be a waste of resources.

The frontier work is on labeling and training expert content, by experts. It's unglamorous work and almost certainly doesn't warrant FAANG pay, but neither did most of the work that most FAANG engineers were already doing. But it does require competent talent from the expert domain.

Like their peer companies, Meta is still sitting on a huge pool of vetted-as-competant workers from the hiring boom and expert AI training is the most ripe business opportunity in a fragile economy where pretty much every comparable opportunity has evaporated.

Yes, this is not labeling what is an apple and what is a pear. "Annotation" does not do this work justice.

For a coding agent, for example, there is *very detailed* analysis of the turns and ranking of different portions of the conversation.

Adherence or deviation from specific rules matters. Writing quality matters. Expertise in the topic under discussion matters. Having intuition for the tone and beat of a good conversation matters.

Scoring a 15-20 turn conversation can easily take two and a half hours.

Clicking submit does not mean the author is done. Many annotations will be turned back to them by a reviewer to touch up in some way.

This work can be far more mentally taxing than programming, is measured much more by completions more of a timed exercise than SWE.

FWIW, Meta employees would probably make great coding agent conversation annotators. But it is absolutely not SWE and they won't enjoy it (for long).

It's a bold move cotton.

I doubt it'll pay off. Let's face it, the average FB engineer is no better than the average Polish engineer, and the Polish guy is 20x cheaper. If you wanted cost-effective labeling you'd send it to Poland. Or better, Chile.

> Software developers in the US are seriously expensive, using them for data labeling would be a waste of resources.

Zuck basically went to a town hall and explained to his employees that their remaining value to him is as training mules for his AI.

It seems really dumb to tell them that. I assume they’re all feeding garbage to the models now.

They’ve largely filtered their employees for people who have a mental breakdown if they don’t get 110% on a 100% scale or get the top of the top rating because they’ve done that their whole life until they got their first job at the tender age of 25-28 after getting their masters or PhD.

I’ve met too many meta members who have stories about their direct reports or peers who had a crashout because they got Exceeds Expectations and a mid 5 figure raise instead of Greatly Exceeds Expectations with a higher 5 figure raise not because of the money but because of not getting the top score.

I’m pretty sure zuck is being a rational sociopath and realizing he can use the PSC system to get these people to widely work against their best interests due to their ego.

idk... In their own way these types of people who have blinders on to such a degree that they crashout over such things are also sociopaths...

Seems like a side effect of the k shaped economy... our society increasingly doesn't have rewards for normal hardworking people. Given tech has been disrupting blue collar jobs for decades I have a hard time feeling sorry for them. Working at Meta already meant you were chasing a bag knowing the product was more or less social poison anyway. It doesn't seem to me that Zuck is uniquely culpable... maybe he's just the best one at the game they're all playing...

The fact he still cares so much about Meta when he is a billionaire is just like those crashouts. Again, it's the same type of person. He's just better at it than they are.

I have no empathy for the types of people who got layd off from meta who can’t see the similarities between them and the industry they disrupted.

I advocated for people who lost their jobs due to tech disruption to learn how to become software engineer not from a place of superiority but from a place of “this is one of the last places you can be self taught and earn a middle class income”

I am saying this has now hit the tech world and I will hold zuck culpable, although not uniquely since that requires a set of 1, but because he’s in charge of one of the top 10 companies in terms of big tech.

You don’t get to be that rich and that in charge of decisions, without holding responsibility. If you want to claim otherwise, then cool, you shouldn’t complain when we take all the assets you are apparently not responsible for.

I like the part when he “took full responsibility” for HIS Metaverse mistake, and fired everybody else lmao

Gavin from Silicon Valley did it first

Zuck literally said that he wants folks with higher intelligence on the Applied Intelligence team. And the best way to do that was to move folks internally, since they were "intelligent" enough to pass the Meta interviews.

Soooo, yes it is a waste of resources ($$$). But this was the initial intention.

> since they were "intelligent" enough to pass the Meta interviews.

I haven't interviewed with them in almost 10 years. But aren't they doing the same interview everyone else does?

The belief that engineers are not doing anything for x amount of time that could be better spent on other immediately measurable things is as old as the profession itself.

Ironically this vanishes when the tables are turned and we ask for things like better hardware or software. There are plenty of us here with stories of how much effort it took to convince employers that SSDs were worth it when they were new, small, and very expensive.

One of the funniest things is how hard it was to get approval for a $100 software license but now people are being encouraged to burn thousands on tokens.

My favorite one is "NO ACCESS TO PRODUCTION DATA" "it's for agent" "do whatever you want"

see also: we're all developing on min spec Macbook Pros while being encouraged to burn > max spec Macbook Pros cost in tokens/month while still waiting for builds to compile.

I can't give you exact numbers, but this is line with what I'm hearing through the grapevine. Lots of senior managers being converted back to ICs as well.

A lot of people are going to leave as soon as they hit their next vest.

It's only until Cold Harbor is completed.

Then all the engineers will get to rejoin their outies.

I don't know what Cold Harbor means in a Meta context, but its interesting that its named after the battle that exemplified Grant's strategy of attrition during the American Civil War. I suspect it means waves of engineers ground down against the defenses of OpenAI/Anthropic in the hopes of eventually finding a crack. Might be best to get out while you can.

> I don't know what Cold Harbor means in a Meta context

Cold Harbor is a reference to the TV show Severance.

Without going into any real spoilers it was the code name of a data classification project so mysterious that the people working on it weren't allowed to know what they were working on (and yes, the project in the show was probably named after the battle in the Civil War).

The Meta connection is that there are some humorous parallels between that project and a project involving people tagging data to train technology to replace themselves, and just the overall creepy dystopian vibe of both the fictional and real-world companies (and founders) involved.

Thanks for the explanation. Meta indeed.

My partner works there as an engineer. The org they work in had loads of people transferred to the "AAI" org doing data labelling. I find it almost unbelievable as well, but it is true.

It's 100% unbelievable and hysterical that its true. Have they done the simple math on how much labeled data they need to make a model of value?

They are likely managing them out.

This is starting to feel like a Peter Watts short story.

I totally agree, it sounds unbelievable… Problem is, I’m on one of these core infrastructure teams, and for my team at least we lost between 50-75% of our engineers to the AI org. Most of the other infra teams I collaborate with have a similar story

Your team lost 50-75%? Is that an approximation, or do you just not know? Can't you just count who left to get the actual ratio?

Maybe it's better to be vague in a situation like this?

Deliberately vague, half so I can’t be easily identified (though my team’s case is not unique), half so I don’t get in trouble for sharing specific personnel information.

True number is much closer to 75% than 50%.

>using them for data labeling would be a waste of resources

Would it? It seems like they can spend a few months extracting intelligence and "taste" from their engineers then get years worth of it back from the AI.

I wouldn't trust any engineers I know of with their "taste". At best it's a highly skewed view of the world. At worst, it's outright opposite to genpop.

I assume taste was meant in term of coding. "taste" is still often the lacking trait that LLMs have when it comes to code design.

Seriously, what a world that would create.

Unless they collude and hatch a plan to sabotage the LLM training.

Are there any examples of this actually working? I keep seeing this fantasy repeated but have not seen a plausible explanation for how they wouldn't be contibuting to the pile of negative examples which are just as valuable if not more.

https://www.cloudflare.com/learning/ai/data-poisoning/ Data poisoning.

its fantasy

scale ai's value prop was catching people like this

what about just... becoming mediocre? engineers are already infamously lazy at reviewing PRs. how is Meta incentivizing these Data Labelers to give a shit and actually scrutinize the AI-generated code they're supposed to be reviewing? what's the reward structure? what prevents engineers from flagging minor nitpicks all day while they look at LinkedIn?

Probably forcing them to review each other's work to panopticon "quality," and keeping track of the average throughput per engineer so if people fall behind the taskmasters can pay them a visit.

It’s just advertisement/SEO. It’s basically guaranteed, just not the way you think

Poison pilling skills is a thing, though finding evidence for it is difficult given the crux is an absence of information. The baseline instruction and training is given to the model by the expert, but edge cases are willfully neglected. The degree of neglect generally determines how detectable it is, but if all the SMEs are in on it a lot of them will probably persist. Effectiveness and impact are obviously relative to the system and the edge case. Not particularly different from the fallout previously seen during the offshoring era.

From the article it sounds like what they're actually doing is reviewing LLM-generated code, for that you do need good software engineers.

Although it goes without saying that good software engineers won't enjoy doing this very much

I believe it, because it makes a kind of sense. Post-training has a huge impact on how well LLMs perform, and labeled data is what determines the effectiveness of post-training. This is why companies like Anthropic are so worried about distillation.

So if you have access to a large number of highly skilled people, and you really don't absolutely need them to do other things, why wouldn't you force data labeling tasks on them?

Facebook is also planning a 10% layoff, so this also works as encouragement for people to leave voluntarily.

(Before you downvote me, note that I'm not endorsing this or saying it's a good idea. I'm just saying that I believe it's true, because I can see how Facebook's leadership would think it's a good idea.)

From the article:

> Forced data labeling with 4,500+ engineers is to generate high-quality RLHF

I doubt that you get high quality from forced reassignments where the now-data labelers don’t actually want to do that kind of work.

It’s crazy to think that Meta leadership believed that it makes sense.

> I doubt that you get high quality from forced reassignments

Their bonuses depend on it. They'll have to play ball unless they have other jobs lined up, are ready to retire early, or prepared to be on the shitlist for the next round of layoffs due to "underperformance"

Do the skills these people have overlap with the skills needed for a good data labeler? I'm guessing being a domain expert is most valuable as a data labeler.

Because you can just get rid of all those people and do the data labeling tasks for 1/4 the cost?

unironically if those engineers were considered to be 'bloat' its better to have them label data because they are smarter and vetted

basically a soft layoff

Isn't that Scale AI investment in a company that does labeling? what are we missing? Are we all going to be labelers soon too?

Plot twist: we already are through the usage of AI lab's API.

Yes

Have you heard about the various startups that specialize in expert data labeling, like Mercor? They can pay $100-$200/hr for highly specialized work, who knows how much they charge their clients. Translated to an annual wage, that's definitely in the SF/SV engineer range

As others have commented, some of the training is very specialized.

Have you heard about working for mercor? You aren't working full time for them or sniffing anywhere close to a proper annual wage.

> using them for data labeling would be a waste of resources

Is work such as changing the styling of a button on the Instagram app any more useful than that?

Mining work was unglamorous and dangerous but was needed. Labeling work is the mining of the AI era.

This reads like a way to get engineers to quit while they work on something "useful". After losing Yann, I doubt its going to end up being useful.

Those percentages seem in line with what I have heard. Not company-wide, MSL was exempted, of course, and probably a few other golden geese here and there.

They are literally doing the apocryphal corporate dystopian maneuver: training their replacements.

They won't be doing it for long.

Coffin building

Silicon Valley strikes again?

https://www.youtube.com/watch?v=obS-qZO9uCQ

I still use the phrase "not a hot dog" to describe things that only does one thing while described as having a lot more capabilities

Not 30-50%.