Slow editing of large papers

#23

Have you ruled out the scheme garbage collector as being the problem?

#24

Nope, I would not know how to do that.

#26

I recorded my experiments. It seems that the issue is not the typesetting, per se, but it is something deeper. The recorded experiments are done with a clean $TEXMACS dir with no configurations. The file is this one: https://savannah.gnu.org/bugs/download.php?file_id=54198 that is attached to this bug
GNU TeXmacs - Bugs: bug #63631, Slow editing of title and author… [Savannah]

I have written a script that sends keystrokes at around 21 keystrokes per second, that is a keystroke every 45ms and types up a filler text.

I experimented with:

  • input outside an environment
  • input inside a proof environment

here are the videos:
Texmacs fresh profile outside environment good performance

Texmacs fresh profile inside environment good performance

Texmacs normal profile inside environment bad performance

Please note how in the text appears in batches in the “bad performance” video. Note that the computer is still typing at constant speed. Also, no indication of anything “wrong” is registered by the benchmark.

I agree that we need to pinpoint a reproducible situation. I spent most of today trying to, but the behavior is sneaky! I am happy that in the experience of multiple people these issues do not occur. Maybe it is because they do not use Windows, Maybe because I often start by importing my files from LaTeX drafts, maybe for some other reason… I do not know.

I often clear my whole $TEXMACS dir and at times it seems to help a bit, but that maybe just placebo. Let me start again with a fresh clean profile. If the sluggishness above reappears, I will record an follow up.

Even with these issues I use TeXmacs regularly, and I have been for more than a year (of course, nothing compared to your expertise). Unfortunately, for me the experience was often a compromise between the great ability to write math naturally and the low responsiveness of the editor. I really hope we can get to the bottom of this (or at least document the things that one should not do!)

Please let me know if you have any ideas on how to continue debugging.

P.S. I was trying to look through my preferences.scm and I see that it gets populated naturally (without me doing any configuration) by lines that look like this:

(“shift-65” “A”)
(“shift-68” “D”)
(“shift-69” “E”)
(“shift-76” “L”)
(“shift-82” “R”)
(“shift-83” “S”)
(“shift-84” “T”)
(“shift-85” “U”)
(“shift-88” “X”)
(“shift-95” “_”)

Do you know what this means?

3 Likes
#27

Would in this case the slow-down depend on something that happens after typesetting?
Besides this, how do you get the debugging output under Windows? Maybe you explained it but I lost it.

#28

@darcy @pjoyez @jeroen some of you can reproduce these problems? or have experienced similar issues?

@guraltsev please don’t get crazy about it. This is the normal state of affairs when dealing with some bugs. A big program is a complex system and its current state is affected by many things, so one has to systematically try to vary some of the conditions to get a sense of what could be the origin of the problem. E.g. what happens if you use article instead of amsart? Or what happens if you remove the rest of the paper and make it shorter. One should try to remove progressively more and more content to see what is going on. As I remarked above I expect the slowness could be originated by a series of conditions which have not been happening often, otherwise it would have been detected sooner. That happened to you in your first use of TeXmacs is somehow random, we just have to understand what is going on and see if we can fix it. Also removing all the $TEXMACS_HOME_PATH is a bit radical. I do not think fonts are important in this problem, so you could try to remove one of the other subdirectories to see what happens.

#29

The benchmarks I run are through WSL (linux under windows). You are right that the windows one does not display debug information.

#30

You are right. I got a bit carried away yesterday. Now that I have a way to debug slowness I’ll report more in detail when I catch some degraded performance.

1 Like
#31

I think one can still run the TeXmacs executable from the command line in a Windows terminal. This should allow to see the output.

#32

Under windows nothing is displayed in the command line. I think I reported this some time ago and it was confirmed.

#33

I managed to reproduce the issue on a completely fresh profile and in the first environment of the paper. (still, a long paper). The typesetting benchmark is:

TeXmacs] std-bench, Task ‘typeset’ took 7 ms
TeXmacs] std-bench, Task ‘typeset’ took 7 ms
TeXmacs] std-bench, Task ‘typeset’ took 7 ms
TeXmacs] std-bench, Task ‘typeset’ took 7 ms
TeXmacs] std-bench, Task ‘typeset’ took 6 ms

However those hiccups are still there. Here is the recorded video

If I remove everything in the paper AFTER the environment the hiccups are no longer there.

Here is the link to the file ubht.tm (it is not the original ubht.tm of the bug report). It is using the article style.

P.S.: I also have turned off the antivirus, just in case.
P.P.S.: I also tried to slow down the typing speed to a much more reasonable 380 Characters/32 seconds (approx 12 char/sec) and the problem is still there.

#34

@guraltsev, apologies for the late reply. I would like to help you troubleshoot this issue, but want to avoid the headache you’re currently facing. I went through this issue a while back, made several posts, and found no solution. I’ve just learned to use texmacs for quick reports. For more serious things, I go back to latex. The performance of Texmacs is quite bad for anything but simple documents. I hope this feedback is helpful and is not taken as a negative criticism. I appreciate the capabilities of Texmacs, but have noticed some limitations in terms of performance.

1 Like
#35

You are on Mac, yes? I actually managed to solve most of my problems with the “multiproject” setting, so I use TeXmacs for large things, too!

1 Like
#36

On linux (manjaro). How did you manage to solve them?

#37

  1. I have a main project file called main.tm with a preamble.
    Then, in this main document I do Tools->Project->Use as master

  2. Then I have a file per section with all the same macros in the preamble.
    Then, in the main document I do Tools->Project->Attach master

  3. In main.tm use Insert->Link->Include to include a link to every file (in order)

  4. You can do Document->Update->All to make sure all references work correctly.

  5. You can quickly switch between files through the Project menu

  6. Things that work:

  • No slowdowns as long as the sections are reasonable (30 pages?)
  • reference numbering
  • clicking references to equations in other files
  • reference tooltips (preview-reference package)
  • I am happy

Notes & Bugs:

  1. To make numbering work in each file I add
  <assign|section-nr|0>

with the section number starting from 0.

  1. If you do not want to reproduce the preamble in each project, create a style file with all the macros. It can be project-local (though in windows this was buggy until recently. I think the fix was implemented but it has not been released in the stable version yet). Assign that style to each file.

  2. If you move/rename the master file then opening the subordinate files crashes TeXmacs. The only way to recover is either to rename the main file back or to open the subordinate tm file with a text editor and edit the line

<project|main.tm>

at the very beginning. (I have not yet filed the bug report).

  1. I think once automatic section numbering is implemented and the multi-window workflow lands it will be very nice!
2 Likes
#38

Sorry to insist, but I do not agree with these vague remarks about the performance of TeXmacs. I use it everyday for papers which are ~40 pages or more, full of math (not many tables) and there are no performance issues whatsoever. three of my students wrote thesis with it in one single document, each ~200 pages. As a developer it is a bit frustrating to hear complain without being able to pinpoint the causes. TeXmacs is not generally slow. I strongly disagree. With @guraltsev we investigates his examples and we found out that TeXmacs is slow (up to be unsuable) with a particular style amsmath with some environments and it is a bit slower in some environment in some situations. I’m yet not sure why but it is likely some bad interaction between the macros and the typesetter, and sometimes the garbage collections. If you look in detail to this thread you see that the examples of slowness are quite specific. I would like to have other examples of slow documents to work on, instead of vague remarks. In particular I do not think TeX is faster (or substantially faster).

1 Like
#39

Editing TeX is surely “faster” because one uses text editors, but there is no (known) structured editor for (La)TeX.

#40

Thanks for your comment. I do appreciate your work in Texmacs. However, after getting hounded last time I posted something on this forum (which eventually led to nothing), I have reservations about opening another front. I’ve accepted the fact that I’ll have to adjust to whatever is available and save myself the headache.

#41

I agree with @mgubi, the localized rendering is impressively fast. For the large papers I currently use the solution I described above with multipart documents.

About the examples of good performance. @mgubi did mention the theses and his own paper, but the .tm source for them is not available. This limits the ability to debug performance or to learn how to avoid weird constructs (and maybe file reports about them). Could one share any of such examples? I shared the screencasts and my .tm files in the thread above (also, could anyone please check whether they can reproduce the issues in the screencast?)

I looked at many online sources, and among the not many Texmacs .tm documents I found, none were above 30 pages (lengths where I also usually do not have performance issues).

Does anyone have a large Texmacs documents that they could share? Unfortunately, arxiv does not allow PDF submissions coupled with other source material. It would be great if people could provide a link to personally hosted source for their Texmacs submissions (e.g. in comment section of arXiv submission).

P.S. Here is an example of a very large file (160 pages) with lorem ipsum filler text and some sections that works perfectly on all my computers
large-lorem-ipsum.tm

#42

Is it not easy to generate a document with random text interspersed with images / tables?

#43

It is more complicated than that, I think. For example, I just tried to generate a random lorem ipsum filler text that was 157 pages long and, as long as I make the pagination be “papyrus” rather than “paper” it is as perfectly snappy as a 1 page document. There is some unfortunate combination of environments and maybe math that seems to be responsible for the issue I am referencing.

P.S. Here is an example of a very large file (160 pages) with lorem ipsum filler text and some sections that works perfectly on all my computers
large-lorem-ipsum.tm