Proposal: Re-impl `TM->PS/PDF` in modern programming language

What

To be discuessed, because re-impling it in a new programming language is time-consuming.

You plan to extend/modify the core typesetter?

@jingkaimori 's plan is to extend the typesetter with new feature like: wrapping text around figures.

Or it is just a translation of the C++ code in Scala?

Not just a translation. The main part is a translation.

Why

What is the goal to rewrite completely something which already works quite well?

Because maintaining a C++ codebase is a pain. Re-implementing the typesetter in a modern language is delightful and efficient.

How

  • Learning about how wrapping text around figures is implemented in other open source projects
  • Make the translation from C++ to Scala
  • Implement the new feature like wrapping text around figures in Scala
  • Implement the new feature like wrapping text around figures in C++

Is the plan to use Scala?

I’m worried it might not be fast enough…

So JVM will be bundled, or at least compile time dependency?

I think Rust is a better idea.

It will be a standalone command line tool. Performance is not the concern.

Let me say that I respect the idea that in open source projects everybody should go for what they like to do, maybe innovation comes from roads less often taken. When I started to work on TeXmacs I’ve also been very enthusiastic and wanted to rewrite this and that. Said that, along the way, and mostly by discussing and working side by side with Joris, I’ve changed my mind. I register here some points for “younger” participants which they may want to consider. I consider them myself when I get enthusiastic about something and want to start a new project.

  • Quality requires commitment. 90% of the code requires 10% of the time (and many projects arrive only here). The last 10%, the most important, that which makes a real usable product, require the remaining 90% of the time. Starting a new project without planning this commitment step will lead to dead code. You can go around GitHub and you find a lot of these dead projects. Which are very useful to programmers to learn but less to users to use, or to developer to make money from.
  • All programming languages are Turing complete. People programmed until now with languages like C/C++/Lisp/Fortran and did fantastic things. Tomorrow a new language will arrive and will look much better. TeXmacs exists since more than 20 years and it is written in a very easy to use C++ with a lot of functional features. Maybe makes sense for Mogan to move to a more modern C++ so to be able to use lambdas, for example. It is not clear to me what would be the advantage of a complete rewrite. In another language maybe something requires 2 lines of code instead of 20, but my experience with the codebase is that one spend very little time coding and much more time to think what is the best solution to insert new code in a complex codebase. This will not be simplified by Scala, even if I admit I do not know Scala, wrt. other modern programming languages, so I might be wrong here, but then you should mention which features of Scala, Rust, X would make the code substantially easier to maintain, if you want to convince me that this is a good idea.

  • The goals are not clear: on one side you say it will be a copy of the typesetting code of TeXmacs to be able to perform the TM->PDF translation (which TeXmacs already do),then you say

but if you add new features (because the Scala code is easier to maintain) then this will be at some point incompatible with TeXmacs/Mogan so another typesetting language, without the features which makes TeXmacs unique. It will resemble a lot Typsts… why don’t contribute then to this last project? We should ask what is our lead wrt. other projects like Typst, SILE,Lout, Scribe or SwiftLaTeX, LyX, Scientific Workplace etc… and we have a lot of lead right now.

I do not want to be too critical, as I said I’m the first one which like to experiment with new languages and do rewrites to make things “right”. But I think new arrivals under appreciate the fact that Joris is a very good coder which choose C++ for stability and Scheme for packability. He does not particularly like C++ either, that is why he choose a very small subset to code with and the program didn’t needed major changes in 20 years, which is a very clear proof that the design is sound. In my opinion is very hard that a rewrite will accomplish anything if changes the structure a lot, or at least this is not clear to me. But again, you are welcome to try, I just wanted to warn that will be a difficult task.

3 Likes

Let me mention an advantage of Rust. It is not a “feature”, but a restriction: codes are based on affine logic by default, which would significantly reduce the risk of resource leakage. “Unsafe” codes are then properly isolated.

In my opinion, it is more important to have more machinery to prevent errors, than some fancy features or “functionalities”. It is similar to pure functional programming, but the optimization and performance of Rust are comparable to those of C, thus the binary output is efficient enough.

2 Likes