As it happens, the current number-two article on HN is about a similar consequence of Chinese export controls--a car manufacturer developing electric motors that do not use rare earths:
The incentives around OSS become stronger the further down in the list of market leaders a company is. The #1 company has no particular incentive to push open software apart from a belief that the market is going to be come commoditised anyway. But the 2nd or 3rd largest player has actual incentives to break the market up and remove software quality as a consideration. No #10 may as well not bother with a proprietary option since if they make it a software quality battle they're going to lose each customer 9 times anyway.
Just because the Chinese are running export controls in one market doesn't mean that they're going to close of access to AI. They might, but each market should be considered in isolation.
And it is nearly always hubris - the people making these decisions are surrounded by yes-men who built their whole career pumping up the egos of their superiors.
As a former freelance translator (1986 to 2005, Japanese to English), I have much sympathy for the writer. But I wouldn’t be so confident that AI cannot do professional-level translation.
She writes: “I adapt, I localize, and I find the best way to convey the original message so it makes sense and feels natural. I research terminology. I make sure it’s consistent throughout.”
I’m sure she has other important insights into what enables her to do her job well. The problem is whether or not such insights can be incorporated into an AI-driven translation system, too.
Since early this year, I have been experimenting with a variety of agentic systems for language-related tasks, including dictionary-writing, research on topics in the philosophy of language, essay-writing, and translation. Other than the dictionary [1], I am keeping the results private, so they haven’t been evaluated by others. But my personal assessment is that agentic systems given suitable high-level guidance can be very good at such tasks now.
If I were still freelancing and I had a large translation job to do for a client, here is the outline of the prompt I would give to Claude to get it started:
“Use this private GitHub repository to build a system for translating [genre of text] from [Language1] to [Language2]. The directory samples/ contains examples of the type of document to be translated, high-quality human translations of those documents, and texts in [Language2] that are in writing styles that I believe to be appropriate for this genre of translation. The file guidelines.md contains my general instructions about the needs of my client and my preferences for how you should translate texts along various axes (natural vs. literal, informal vs. formal, preferred dialect in [Language2], consistency vs. variety in terminology translation, etc.). Begin building (1) a knowledge wiki for this project using Karpathy’s LLM-wiki framework and (2) a system inspired by Karpathy’s Autoresearch, AutoResearchClaw, etc. for testing and recursively improving both the functioning of the system and the quality of the translations. For the actual translation, editing, checking, etc., use not only your own ability and the knowledge assembled in (1) but also outsource such tasks to other frontier models through OpenRouter, and use adversarial evaluations among those models and yourself to check and recursively improve the system design, the prompt-writing for other models, and any translations created by the system. My OpenRouter API key is available in this environment. You may spend up to $xx per day in API calls until this project is ready to do real translations; before beginning a real job, give me an estimate for how much the API calls will cost for that job. The initial build-out of this project will take many sessions, so write a prompt called resume-prompt.md that I can point you to at the start of a scheduled Routine to have you work on this. Commit and squash-merge to main at the end of each session. I will be checking in occasionally to view your progress and to ask you to run translation tests, and I will offer guidance then on how to improve the pipeline further and make the translations closer to what my client needs. If you have any questions before you begin, please ask me.”
My first came in late 2016, when Google Translate switched from statistical machine translation to a neural-network-based system. I had worked as a Japanese-English translator and lexicographer for two decades, and I had been testing various machine-translation services over the years. For translation between Japanese and English, at least, they were uniformly terrible: the output for genuine texts was mostly incomprehensible and could not be used for any real-life applications. The neural Google Translate, while still far from perfect, was suddenly useful for some purposes.
But the neural models were still not translating meaning, which is the whole point of translation. I devised a variety of tests to see if GT could identify the meaning of ambiguous words from the context, and it couldn’t. One example I would show people was the sentences “I was born in 1998, and my sister was born in 1999” and “I was born in 1999, and my sister was born in 1998” translated into Japanese. Japanese uses different words for older and younger siblings, but GT translated “my sister” with the same word in both sentences. It was easy to come up with other examples where GT would fail, such as when the meaning of a word could only be determined based on context in a previous sentence; at that time, GT seemed to be translating sentence-by-sentence, with no consideration of what came before or after. I kept waiting to see whether computers would ever be able to handle meaning when translating, and for years thereafter there was little progress.
A minor shock came in mid-2022, when DALL-E 2 was released. Its ability to create images from natural-language prompts suggested that something deeper was going on than just statistical correlations. But I couldn’t see yet what the useful applications might be.
My biggest “oh shit” moment came with ChatGPT in late 2022. While the initial release didn’t translate Japanese well (I seem to recall that there were character-encoding issues), I ran various tests to see if it could, for example, identify the antecedents of pronouns and the meanings of polysemous words in English based on the context. It did really well. Last December, I gave a talk at a university in Tokyo in which I showed some examples done with the 2022-era GPT-3.5. They appear in slides 4 to 8 of the following:
> There is danger in evaluating for language patterns over its content
I agree, but it’s worth noting that that has been done since long before LLMs. Fifteen years ago, I used to teach a graduate course on academic writing pedagogy. The students and I would read research papers on the teaching of academic writing; we also analyzed textbooks and course syllabuses to get an idea about what was actually being done in classrooms. While phrases like “critical thinking” did come up, the overall focus was clearly on language patterns: sentence and paragraph structure, the use of transition words, vocabulary for hedging and boosting (i.e., making assertions seem weaker or stronger), etc.
In a university context, it can be very difficult to evaluate student writing based on its content. In humanities-focused and creative writing, what the student decides to say can be seen as an extension of the student’s personality, identity, and individual experience; if a teacher evaluates the content, including the reasoning, it can seem that the teacher is evaluating the student as a person. And if the students are in the sciences, especially at the graduate level, the writing teacher often won’t even understand what the students write because it is too technical. Teaching and evaluating language patterns, not content, is often the only option.
The focus of the course wasn't writing criticism or particular authors. It was on how writing teachers can teach current and future researchers how to write research papers. A narrow field, perhaps, but at the research university where I worked there was demand for such writing courses, and my class looked at how such classes can be taught at a meta level. Many of the students themselves became writing teachers.
This was all before LLMs. The assumptions behind my class have since been upended, now that easily available chatbots can produce academic writing that, at least at the structural level, is perfectly fine.
Since January, I’ve been having Claude build a static Japanese-English dictionary in which all of the kanji and jukugo can be displayed either with or without furigana:
I haven’t spotted any mistakes in the furigana myself, though there must be some. I have a scheduled routine running multiple times a day to have Claude check and polish existing entries; it should be correcting most of whatever furigana mistakes might be in the data. At some point, I will set up an agent to use a different LLM to run a similar set of checks to try to reduce the error rate even more.
As you note, the readings of Japanese words depend on the context, so producing accurate furigana cannot be done entirely programmatically. Sentences must be interpreted semantically.
I am releasing all of the dictionary data into the public domain, and anyone is free to fork it or adapt it however they like:
Thanks for sharing this. It looks like a really cool project, and making the data public domain is especially generous.
I especially like the dictionary + example sentence format. I haven’t found a really good Japanese-English dictionary for learners, and yours looks promising.
I’m curious how token-intensive the repeated Claude polishing runs are.
I was curious whether the distribution would vary from model to model. Here are the results for 1,000 queries each for smaller models in the Gemini, Mistral, Qwen, DeepSeek, and GLM series:
I really like working from home myself, but I am starting to suspect that the organizations that thrive in the years ahead will be those with lots of face-to-face interaction. If most people are working remotely and having AI agents communicate on their behalf, trust won’t form, consensus-building and decision-making will suffer, and employee salaries will start to seem like a waste of money.
His LLM-wiki framework has been very useful for me for some personal research and knowledge-building projects I've been working on recently. When I get an idea for a new project, I first give it to Claude together with LLM-wiki.md and have it spend a few sessions compiling knowledge in the wiki before beginning work on the project itself. I schedule further wiki-maintenance sessions for later, too. Over time, the wikis become especially valuable when planning major changes or additions to the projects, as they help to ground both me and Claude with knowledge specific to the project.
Here's an example wiki in a public repository for a dictionary I have been having Claude build for the past few months:
I’m teaching a class at a university in Japan (on AI-related issues, as it happens). I’ve been teaching for more than 40 years, but at 106 registered students this is by far the largest class I have ever taught. AI tools are very helpful for class management, such as keeping track of attendance and homework submissions.
I have to consciously avoid using AI for more cognitive tasks, though. It would be very tempting to have Claude, ChatGPT, or Gemini summarize, classify, and grade the students’ assignments, write individual feedback, prepare my lesson plans, etc. However, I know that my engagement with the material and with the students would suffer. I also want to show the students that they are learning together with me and with each other, not with bots.
I am semiretired and have a light teaching load that gives me plenty of time to prepare for class. I can see that full-time teachers might find it hard to resist the lure of offloading their thinking to AI.
The image produced by the program seemed unbalanced because Japan’s southernmost islands were included even though they are not part of the electrical grid. I used an image editing program to remove the outlines of those islands and shift the main part of the country toward the center.
Side comments:
Not indicated on the map is the fact that Japan’s electrical grid runs at 50 Hz in the eastern and northern parts of the country and 60 Hz in the west:
https://news.ycombinator.com/item?id=48510010
reply