Unicode in verbatim code

I want to put some Julia source code listings I have that use unicode characters. I have generated my initial version as a conversion from markdown to latex, then to texmacs. Inside the .tm file the source code looks all right but inside texmacs and inside the exported HTML it has been mangled. For example, this is the original source code with some greeks letters here and there:

struct SGD{T <: AbstractFloat, N} <: Optimizer{T, N}
    θ::Array{T, N}
    ϵ::T
    SGD(θ::Array{T}, ϵ::AbstractFloat) where {T <: AbstractFloat, N} =
        new{T, N}(θ, convert(T, ϵ))
end

This is how it is represented inside the texmacs file inside a <\verbatim>:

    struct SGD{T \<less\>: AbstractFloat, N} \<less\>: Optimizer{T, N}

    \ \ \ \ θ::Array{T, N}

    \ \ \ \ ϵ::T

    \ \ \ \ SGD(θ::Array{T}, ϵ::AbstractFloat) where {T \<less\>:
    AbstractFloat, N} =

    \ \ \ \ \ \ \ \ new{T, N}(θ, convert(T, ϵ))

    end

    \;

And this is what I get inside texmacs and inside the generated HTML:

struct SGD{T <: AbstractFloat, N} <: Optimizer{T, N}
    Îÿ::Array{T, N}
    Ïţ::T
    SGD(Îÿ::Array{T}, Ïţ::AbstractFloat) where {T <: AbstractFloat, N} =
        new{T, N}(Îÿ, convert(T, Ïţ))
end

Copy-pasting from the original source code inside texmacs also gives errors:

As I understand Texmacs doesn’t use Unicode for .tm files. I wouldn’t mind making a script that converts my source code listings to whatever representation uses texmacs for unicode but I have not found any info inside the help system.

Cheers,
Pedro.

Can you share the latex file here?

Hmmmm… The type of file is not authorized to be uploaded here it seems and I think maybe it’s too long to be copy-pasted? I have created a gist file, here it is the link to the line that first uses unicode chars:

The original blog post is here just to compare: https://plopezadeva.com/2020/11/08/julia-first-impressions/

Thanks!

It seems to me that the code where you have the red epsilon is the correct representation, if you export that in HTML you obtain

<pre class="verbatim" xml:space="preserve">
abstract type Optimizer{T, N} end

struct SGD{T &lt;: AbstractFloat, N} &lt;: Optimizer{T, N}
    &theta;::Array{T, N}
    &#x03F5;::T
    SGD(&theta;::Array{T}, &#x03F5;::AbstractFloat) where {T &lt;: AbstractFloat, N} =
        new{T, N}(&theta;, convert(T, &#x03F5;))
end

function step!(optimizer::SGD{T, N},
               g::T,
               i::Vararg{Int, N}) where {T &lt;: AbstractFloat, N}
    optimizer.&theta;[i&hellip;] -= optimizer.&#x03F5; * g
end
</pre>

which, as far as I can see is correct. The only problem is that somehow the unicode point is represented as a string, like if the corresponding glyph was missing in the font (which I do not understand why). So I think is mostly a problem with the mapping to glyphs than a problem with the conversion. But it is true that the import from LaTeX is not correct. Somehow it seems to me that TeXmacs does not recognise the file as unicode.

Yes, my impression is correct. If you change the document font to “Stix” then I see

while with “Computer Modern”

Btw, I’ve tried to copy the HTML from the webpage and paste in TeXmacs and it works fine. Only the formulas are a bit scrambled. But the code is imported correctly. Usually is better to write directly in TeXmacs. If your original file is in Markdown there should be somewhere some attempt to have a Markdown converter. Or you can use pandoc to export Markdown to a nice format like HTML and then import HTML. I would avoid to go thru LaTeX if not necessary.

OK, thank you! I will change fonts to Styx and copy paste directly inside Texmacs.

This is the last roadblock I think to convert my little blog to Texmacs. It is not 100% what I want yet but I’m willing to compromise because for my use case, a blog that acts as a set of notes, I usually think as I write and it’s very helpful to see the final result. I hope to publish it this weekend, I will add a link to this conversation so the results can be seen.

Thanks again @mgubi.

Looking forward to see the result! I think you can use other fonts besides Stix, like Termes for examples. I use often Biolinum. My experience managing the TeXmacs’ blog is quite positive so far. I think we can improve the output but much can be done right away by using tmhtml-* tags and CSS. We could write some tools in scheme to manage blogs specifically. An example of static site generation with scheme is https://dthompson.us/projects/haunt.html and also https://wingolog.org/projects/tekuti/ . We do not need much infrastructure since all the editing can be done in TeXmacs and then export to HTML, but we might want to add some automatic index generation, feed generation, etc…

OK, if anyone is interested I finally managed to migrate. I adapted also @mgubi feed generator for my purposes. Anyway the final result can be viewed at https://plopezadeva.com and the source code is available at this repo (I was feeling alternative and wanted to try sourcehut). Cheers.

1 Like

It looks nice. I think a smaller font would make it easier to read (I have a 19’’ monitor and too little text fits into it).
I agree on your opinion that TeXmacs would benefit from a larger number of active users—if this were true, there might then be also more developers, and almost by definition there would be more functionalities (since the program can be extended through Scheme).

Yes, very nice. I also find the font a bit too large. Is intentional? This is how it looks on my screen:


Looking forward to read your articles and also suggestions on how to improve blogging experience with TeXmacs.