there are models between 2-grams and 600m param models that would be good options. i don't expect a 2-gram to do very well here. also i'm not sure why this model isn't a fine choice if it solves their problem

What would you suggest instead?

A non-autoregressive transformer trained with a classification objective.

These are absurdly effective for this kind of task. Training is fast and straight forward. Packaging for deployment as ONNX is pretty simple as well.

As a follow up to the original article, I added a new experiment using Logistic Regression and the results are very good. It actually improves on the accuracy by a few points.

More details here: https://www.teachmecoolstuff.com/viewarticle/using-logistic-...

you should link your data to wikidata which will get you free connection back to crunchbase and other sources e.g. https://www.wikidata.org/wiki/Q97041185

You could even back some of the data from there

EU tried to ban tracking cookies. Anyone could've seen the result coming (every site now disrupts your reading). The defeatist attitude would've been right there.

I don't see how games won't just charge you a $0.01 "subscription" that lasts 5 years or various other such sidesteps. it'll just make everything more annoying.

anyone have a free photo for wikipedia?

Using mtp takes away some of the magic for me

That was my reaction too. Most of what I wanted to know from the video is how it would emulate a filesystem that does that.

i assumed it's for influencers who want to make a video of the new city or something

Amusingly you just conflated the pico (a dev board) with its chip (rp2040)

Then maybe caveat your posts with that

yeah i really don't like the corpus of statements and it makes me doubt lenz. consider

> “Artificial intelligence will cause widespread job loss among software engineers.”

https://lenz.io/c/ai-software-engineers-job-loss-impact-05e4...

this is a statement about the future. who knows? dataset also includes

> Robots will not replace human teachers in schools in the near future.

or

> Papua New Guinea has very few female members of parliament.

what counts as very few?

> “Taurine supplementation supports mood and emotional health in humans.”

why is this labeled as misleading? i'm not even sure when I'm supposed to use the misleading label

> Anaximander was the first scientist in recorded history.

this is a judgement call as the term scientist didn't exist.

the claims that feel actually solidly answerable seem to have much better LLM performance

Agree that some of the claims are forward-looking. The messiness of the real-world and real-user fact checks. No ground-truth verdicts are provided or used in the study though. It only measures the level of agreement between the selected models, not which one is right on which claim. I.e. none of the claims is actually labelled.

were you involved in making the study? your bio says you work for them so you should probably indicate that in your comments.

lack of agreement when there is no singular correct answer (or any answer at all) isn't a useful metric

I ran into a lot of these kinds of issues when working on the Citation Needed WMF project (and related extensions). Truth is so often very nuanced.

They introduced themselves as the study author here: https://news.ycombinator.com/item?id=48307887#48307899

ah. I missed that.

Most jeeps never go off-road