Bug in the pdf-import

Tilda · February 2, 2022, 10:52am

I did pdf-plots within the Maxima-plugin containing Unicode math characters such as “⅓” and “⋅” for the “key” labels. Maxima and gnuplot produce the correct pdf output, e.g.:

However, when I want to load these files into my tm-file these Unicode characters are only partially imported by TeXmacs (2.1.1 on OS X 13.6/15.7) and the function graphs not rendered at all:

Does anyone have an idea why this isn’t working correctly? Could it be that Texmacs’ pdf-renderer is flawed?

Exporting the tm-file to pdf via preview seems to be working correctly, i.e. correctly include the pdf-plots.

By the way, saving the gnuplot plot as an svg-file and then loading it into TeXmacs didn’t really solve the issue either – the legends disappeared completely, but at least the graphs would have been drawn.

Any help appreciated.
Cheers, Tilda

mgubi · February 2, 2022, 1:16pm

AFAIR the pdf rendering is done by Qt, so it looks like a bug there, when we export to PDF we use our own renderer (which in this case I guess only include the PDF for the image) and this is the reason the same problem does not appear. What happens if you use some PDF viewer based on libraries like Poppler?

Tilda · February 2, 2022, 2:13pm

Thanks for the fast, clarifying reply, mgubi.

Do you think that this Qt issue will be fixed/gone when TeXmacs will be using Qt5?

Concerning your last suggestion, as I do neither have problems viewing these pdf-files with external pdf-viewers nor printing the tm-files using “preview” using the default settings, I’m not sure how your hint might help.

Cheers Tilda

mgubi · February 2, 2022, 2:49pm

I do not have any idea. When I have some time I will look into it, it would be useful if you file a bug report for TeXmacs anyway with the PDF file which causes the problem. Which PDF viewer have you tried? Qt uses an external library (I think) to read PDF, so if there is a bug it mush show up in other programs which use the same library.

Tilda · February 2, 2022, 3:30pm

Thanks for your cooperation, dear mgubi.

Next to Apple’s preview and Adobe’s acrobat, these pdf files are correctly displayed by

xv 3.74
zathura 0.4.4
mupdf 1.17.0

Whether these three programs use the poppler library I would’t know. I’m more than willing to file a bug report but I have no idea where to and how.

Cheers
Tilda

jeroen · February 2, 2022, 5:23pm

@Tilda Bugs can be reported here:
https://savannah.gnu.org/bugs/?group=texmacs

mgubi · February 2, 2022, 9:52pm

I’m not sure which library Qt uses, maybe xpdflib or poppler. But I think it is safe to file a bug report with the offending PDF file so that we can investigate the matter further. If it is a problem with Qt I fear there is not much we can do in the short term, but we will see. The bug tracker is here: https://savannah.gnu.org/bugs/?group=texmacs

pjoyez · February 3, 2022, 8:52am

I believe that in your case TeXmacs used ghostscript to interpret that pdf (like Max said, Qt has no built-in pdf interpreter).
Please try rendering the drawing with gs from a terminal and check if it complains about issues in the pdf file. In that case, it would be a gnuplot bug, to file with them. The reason your pdf viewers nevertheless display the file correctly is because many pdf viewers go a long way trying to cope with malformed pdf.

Regarding svg, at this point TeXmacs handles it through Qt, but Qt’s svg support is very disappointing. I believe we can improve on that soon by using good external converters.

mgubi · February 3, 2022, 10:02am

Thanks Philippe. Indeed this morning I was trying to look into the code to see what we do exaclty and I found the situation a bit disappointing. I’m now convinced that we need to have our own code to handle PDF and some other image format, like PNG and JPG at least and do not rely on Qt or any other external library. This is what we do already for PDF output, but also input is important for TeXmacs and coherence of the results among different OS. I will think what is a good library to embed in our codebase. For SVG I do not know what to do, but we could imagine to have our own redenderer.

Tilda · February 3, 2022, 6:44pm

@pjoyez I rendered the pdf-plot using gs filename.pdf. gs/Ghostview 9.54.0 displayed these pdf-file without complaints and absolutely correctly. If I understood you right, we can exclude a gnuplot bug now.

As @mgubi suggested after investing the code, fixing the issue of correctly importing various image formats into TeXmacs seems to be a larger, but necessary, project.

I’m going to try filing a bug report to https://savannah.gnu.org/bugs/?group=texmacs this weekend.

Cheers
Tilda

Tilda · February 4, 2022, 9:28pm

@mgubi I just filed the bug report as you suggested. Tilda

jeroen · February 4, 2022, 9:44pm

Thanks for the report. It’s very odd, I can’t reproduce the bug with the your pdf file. Which version of TeXMacs are you using?

Tilda · February 4, 2022, 9:46pm

TeXmacs 2.1.1 on osx 13.6/15.7

jeroen · February 5, 2022, 12:31pm

Perhaps someone can try on Mac.

How did you add the image into your document? Does it make a difference if you link or insert the pdf?

Tilda · February 5, 2022, 1:04pm

I’m afraid it doesn’t make any difference what so ever.

mgubi · February 5, 2022, 2:01pm

Things are even more strange: on my Mac the following happens, the version of TeXmacs which I downloaded from the website show the bug which @Tilda reported. The version of TeXmacs which I compile myself from the svn repository doest, it shows a correct rasterisation of the PDF file. I need really to look into the code to see what in the environment influence the way we handle the rendering of PDF files.

Tilda · February 5, 2022, 7:46pm

@mgubi thanks for your efforts. @pjoyez I ran TeXmacs in debug mode as you suggested on savannah but I some how fail to upload my comment including the output of my debug run.

Tilda · February 5, 2022, 8:05pm

@pjoyez It’s in savannah’s comment#2 preview but it doesn’t upload. So, for the time being I’ll post it here:

This is a preview

I ran /Applications/TeXmacs.app/Contents/MacOS/TeXmacs --debug-convert bug_report.tm

This is the output:

/Users/matthias/.TeXmacs/progs/my-init-texmacs.scm:511:1: In expression (assign “darkgreen” (macro “body” #)):
/Users/matthias/.TeXmacs/progs/my-init-texmacs.scm:511:1: Unbound variable: assign
TeXmacs] debug-convert, Loading dictionary corktounicode
TeXmacs] debug-convert, Loading dictionary cork-unicode-oneway
TeXmacs] debug-convert, Loading dictionary tmuniversaltounicode
TeXmacs] debug-convert, Loading dictionary symbol-unicode-oneway
TeXmacs] debug-convert, Loading dictionary symbol-unicode-fallback
TeXmacs] debug-convert, Loading dictionary symbol-unicode-math
TeXmacs] debug-convert, Loading dictionary corktounicode
TeXmacs] debug-convert, Loading dictionary unicode-cork-oneway

TeXmacs] debug-convert, Loading dictionary tmuniversaltounicode
TeXmacs] debug-convert, Loading dictionary unicode-symbol-oneway
TeXmacs] debug-convert, Loading dictionary corktounicode
TeXmacs] debug-convert, Loading dictionary cork-unicode-oneway
TeXmacs] debug-convert, Loading dictionary tmuniversaltounicode
TeXmacs] debug-convert, Loading dictionary symbol-unicode-oneway
TeXmacs] debug-convert, Loading dictionary symbol-unicode-math
TeXmacs] With linked TrueType support
TeXmacs] debug-convert, image_size not cached for :/Users/matthias/Documents/progs/maxima/mea/Dowling07/bug_report_sample_05.pdf
TeXmacs] debug-convert, hummus_pdf_image_info:
TeXmacs] debug-convert, image =/Users/matthias/Documents/progs/maxima/mea/Dowling07/bug_report_sample_05.pdf
TeXmacs] debug-convert, crop box={0, 640, 0, 384}
TeXmacs] debug-convert, w,h={640, 384}
TeXmacs] debug-convert, degrees image rotated :0
TeXmacs] debug-convert, dx,dy={-0, -0}
TeXmacs] debug-convert, image_to_png … using gs
TeXmacs] debug-convert, gs_to_png using gs
TeXmacs] debug-convert, image_size in cache for /Users/matthias/Documents/progs/maxima/mea/Dowling07/bug_report_sample_05.pdf
TeXmacs] debug-convert, 640 x 384
TeXmacs] debug-convert, w=620 h=372
TeXmacs] debug-convert, bbw=640 bbh=384
TeXmacs] debug-convert, res =69 * 69
TeXmacs] debug-convert, “/Applications/TeXmacs.app/Contents/Resources/share/TeXmacs/bin/gs” -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=pngalpha -dGraphicsAlphaBits=4 -dTextAlphaBits=4 -g620x372 -sOutputFile=/Users/matthias/.TeXmacs/system/tmp/18417/tmp_845492656.png -r69x69 -dUseCropBox /Users/matthias/Documents/progs/maxima/mea/Dowling07/bug_report_sample_05.pdf
TeXmacs] debug-convert, answer :GPL Ghostscript 9.24: Some glyphs of the font YMVOSG+Helvetica requires a patented True Type interpreter.
TeXmacs] debug-convert, **** Error reading a content stream. The page may be incomplete.
TeXmacs] debug-convert, Output may be incorrect.
TeXmacs] debug-convert, **** Error: File did not complete the page properly and may be damaged.
TeXmacs] debug-convert, Output may be incorrect.
TeXmacs] debug-convert,

What I do not understand is that TeXmacs calls Ghostscript 9.24. My default Ghostscript is version 9.54.0 as setup by MacPorts.

Tilda · February 6, 2022, 11:48am

@pjoyez Dear Philippe, thanks for confirming the bug report on savannah –

status confirmed: outdated gs in MacOs .app => pdf issue

In the meantime, until a new release of TeXmacs is issued, how can I make the TeXmacs.app ignore its native gs 9.24 and instead use my default gs 9.54.0?

Cheers, Tilda

mgubi · February 6, 2022, 3:25pm

The hard way is to copy your version of gs over /Applications/TeXmacs.app/Contents/Resources/share/TeXmacs/bin/gs or replace this last file with a symlink to the “good” gs. I’m not sure if there is a way to tell TeXmacs to use a different gs.