I wonder what is the state of art of ChatGPT / codex integration within TeXmacs? It would be useful, given the current strength of ChatGPT in theoretical science.
ChatGPT / codex integration
1 Like
Probably zero right now, but you may have just given me the final push I needed to start developing some sort of integration with Codex myself.
What I was thinking about is something like the VS Code right-side panel that pops up, but with a much simpler interface. It would essentially spawn a terminal in the current document’s directory, start Codex or Claude Code, and give the agent an AGENTS.md file telling it exactly how it should interact with TeXmacs documents.
I actually have quite a bit of experience with agentic models editing TeXmacs source files, and unfortunately they struggle quite a lot with the syntax. Sometimes they start using LaTeX syntax inside TeXmacs files and break everything. Another common problem is with closing tags/callouts or mathematical expressions in TeXmacs syntax: they forget to close something, and the whole document breaks.
I mostly solved this by writing a small Python script that acts as a sort of linter and checks the structural integrity of the document. Simply making the agent run this script after it finishes editing is enough to reduce the error rate dramatically.
Even with this, though, the model still manages to wreck a document maybe one time out of seven. So you still have to manually check its work and, if it did something wrong, either fix it yourself or ask the model to correct it.
Another really annoying thing, and probably one of the main blockers, is encoding. TeXmacs files use Latin-1 encoding, while these models generally expect to work with UTF-8. So basically every time the model wants to modify something, it has to convert the file to UTF-8, edit it, convert it back to Latin-1, and replace the original file with the modified one.
This is actually relatively fast and could probably be handled by a script, but it adds another layer of complexity. More importantly, it means that the changes the model makes aren’t really live: you basically get the result only at the end of the turn.
And if something in that conversion goes wrong — for example, if the model introduces a character that cannot be represented in Latin-1, which has happened to me — things can break badly. In the best case, TeXmacs just shows you the usual red encoding error because it cannot load a character. In the worst case, you open the document and get an empty document because TeXmacs doesn’t know what to do with it, and you have to revert to an older version.
So any potential Codex extension would absolutely need a Revert changes button and some sort of version tracking, so that you can undo whatever mess the model just made.
From experience, the most reliable workflow I’ve found is to make the model export the document to PDF after editing and then visually inspect the pages it modified to check whether anything broke. This dramatically reduces the error rate.
The problem is that it also consumes a lot more tokens and takes much more time. It becomes painful very quickly once your documents get even moderately large and take longer to render to PDF.
And that leads to another issue: TeXmacs keeps its own buffer of the document in memory. So if an external agent modifies the file on disk, TeXmacs does not automatically show those changes. You have to reload the document from disk.
That means that, at the end of every agent turn, something needs to tell TeXmacs to reload the current document from disk.
I’m not sure how to do that cleanly. I’m not sure whether TeXmacs exposes an API for it, but I don’t believe it does.
So there is a chance that this would require modifying TeXmacs itself. At that point, though, the Codex integration would no longer really be just an extension, because users would have to recompile TeXmacs or download a modified version that supports this functionality.
Of course I could make a pull request to the TeXmacs repository with the small change required for this, but I have no idea whether it would be accepted, or how long that would take.
And the list of issues actually goes on.
If you want an AI model to modify a TeXmacs document, you probably also have to change the way you use images.
Unlike LaTeX, TeXmacs has the very convenient ability to embed images directly inside the document. That means you can have a completely self-contained document consisting of a single file, which is fantastic for portability.
The problem is that, from the model’s point of view, the TeXmacs source file then contains hundreds or thousands of lines of what looks like complete gibberish but is actually encoded image data.
That obviously confuses the model, consumes a huge number of tokens because it has to read all that data, and increases the probability of it accidentally modifying or breaking something.
Fortunately there is a simple solution: link all embedded images instead. Then the actual image files remain external to the document and the TeXmacs source becomes much cleaner and easier for the model to understand.
But the obvious compromise is that you lose the single-file portability that made embedded images so convenient in the first place.
And this next part is more of a rant than an actual blocker for a Codex extension, but TeXmacs has hardcoded the “link all embedded images” functionality so that it puts every linked image in the same directory as the document.
You can’t do something like Obsidian, where all the images automatically go into a subdirectory such as Attachments/ .
Instead, you’re forced to dump every image directly into the document directory, which creates an absolute mess. And it is so ugly. I cannot stress this enough.
To change this behavior so that TeXmacs would put linked images into an Attachments/ folder, I literally had to modify the TeXmacs C++ source code and recompile it from scratch.
That is absurd. So many little behaviors like this are hardcoded deep inside TeXmacs.
So yeah, that’s basically the gist of it.
If anyone wants to build a proper Codex/agent integration for TeXmacs, these are probably some of the main problems we’ll run into. And I’m probably forgetting a few.
There is some support for interacting with LLMs in recent TeXmacs versions. See “Help -> Plugins -> Ai”. I believe the way it works is to inject a request to format the response as LaTeX and then use TeXmacs builtin LaTeX -> TeXmacs conversion. I don’t know if it handles images.
I’m quite happy about ChatGPT working on TeXmacs files, just ask him to use a custom patcher which understands Cork encoding, it mostly works and certainly can be improved easily. Check my website (https://github.com/mgubi/mgubi.github.io) which is now maintained via an agent.
1 Like
Depends a lot on your workfow.
Your fix would be very useful to me if it was just for small edits here and there.
But to me it would be actually worse.
I use agentic models to review big chunks of documents, correct and add sections. So a big reading and writing workload.
With the helper, a multi-line change has to be written as JSON with \n and \<alpha\> escapes. Every extra layer of escaping is a chance to get a backslash wrong, and trust me, these models get them wrong a lot on long edits.
Plus, the model is not forced to see a bunch of � stuff while trying to read accents…
Genious actually, really like this feature
If I understand correctly, one has to set up the key via environmental variables. I am not sure whether this is a good way to do it.