Contribution (of plugin) towards a scientific note-taking system

#1

Hi all,

For the past half year I have been exploring TeXmacs. More and more am I finding myself surprised by the versatility and inner elegancy of this program. I would like to try to create an extension to fill gaps between TeXmacs and wiki style note-taking systems (Roam Research, Obsidian).

I would like to know if it is actually feasible and what will be the expected difficulties (see proposed functions below) … Thanks!

Rationale

I have always wanted to find a way of notetaking that fits my 4 core requirements, namely:

  1. It must allow me to type in almost verbal speed. (this eliminates LaTeX*)
  2. It must be truly WYSIWYG. (this Eliminates Obsidian (sorry), VNote, and basically most md-based notetaking system)
  3. It must be open source, with storage done primarily offline, and does not use Electron bloat (this eliminates OneNote, Notion)
  4. It must have good typesetting, with support over formulas. (this eliminates all rest xD)

Before TeXmacs the one that mostly fits all 4 above is Zim wiki. With configurations it performs amazingly and is very extensible. However the typesetting is rather primitive. Also due to some upstream issue there are various UX bugs that were not fixed for years.

TeXmacs fits all 4 extremely well. And it has elegant core logic, is extremely extensible and just keeps surprising me in the great documentations and various style packages. But unfortunately it is intended as a scientific document editor, hence does not support well over the multi-document wiki scenario. I have been writing some scripts externally to help me manage the files and filling the missing pieces, but it is certainly better if it can be integrated into TeXmacs itself.

Proposed functions

(1) A treeview file browser
(2) Support for structured “search everything”
(3) Support for interlink between files and preview-ref between them

Again, I am amateur at Scheme stuff, so don’t expect too much of me :joy: But I’ll try to see if there is something I can do and will reply in this thread when there is progress. I would be even happier if there is some help I can get. Thanks xD

6 Likes
#2

Welcome to contribute to https://github.com/XmacsLabs/mogan

We are developing on it heavily. And this summer, because of the OSPP event, we will try to write documentation for new code contributors and attract more contributors.

1 Like
#3

Two tools that may be useful to implement this sort of thing are user databases (similar to the internal bibliographic database) and the tmfs filesystem. There could be a database of notes with tags, folders and unique ids which can be used to link notes together. Some tmfs handlers could then be defined to display, for example, all notes with a certain tag, a folder tree structure or a graph of linked notes.

tmfs is well described in the documentation. User databases not so much. I have done some experiments with it and may be able to help, if you want to go down this route.

4 Likes
#4

Hi,
TeXmacs is quite good, out of the box, for note taking and wiki style (or zettelkasten) operation. For example you can check our the blog (https://github.com/texmacs/notes) or my website (https://mgubi.github.io/docs/main.html) which are essentially multipage wiki-style interlinked TeXmacs documents which can be edited and browsed in TeXmacs and then exported to HTML for external browsing. What is currently missing is some tagging and metadata handling mechanism, global search and some automatic generated pages which comprise all the documents. All these things should be quite easy to realise in scheme. For example this page: https://texmacs.github.io/notes/docs/list-articles.html is generated automatically from the sources via a scheme script (https://github.com/texmacs/notes/blob/main/notes-tools.scm) and some more indication on how to edit or scrape informations from TeXmacs documents are here (https://mgubi.github.io/docs/zettels/scripting-webpages.html).

@jeroen suggested to use tfms which is a good idea, especially to interface to search and create an API to access pages.

3 Likes
#5

Some more useful pointers:

  • how search is implemented for TeXmacs documentation (tmdoc style): see $TEXMACS_PATH/progs/doc/tmdoc-search.scm

  • how to generate programmatically documents: see e.g. how Joris generate the documentation for specific focus loci, e.g. top level documents in $TEXMACS_PATH/progs/generic/document-doc.scm

3 Likes
#6

Wow, thanks!

For example this page: https://texmacs.github.io/notes/docs/list-articles.html is generated automatically from the sources via a scheme script (https://github.com/texmacs/notes/blob/main/notes-tools.scm ) and some more indication on how to edit or scrape informations from TeXmacs documents are here (https://mgubi.github.io/docs/zettels/scripting-webpages.html ).

This is very helpful :rofl: The “wiki home page” part is already implemented there. Maybe I will try to expand it into sort by date mode, by tag mode, and original treeview mode. First two should be easy.

  • how search is implemented for TeXmacs documentation (tmdoc style): see $TEXMACS_PATH/progs/doc/tmdoc-search.scm

I feel the search everything function is more closely related to doc/docgrep.scm (the one in Help->Search->Documentation...). Basically the grep can be used out of box xD although it does not seems to highlight the words in individual documents by default lol

For search everything I prefer to let the results show in popup / side pane instead of in current buffer, as that is distracting. I am currently considering to use docgrep etc. as backend, then feed data to rofi for external popups. The coupling will be minimal and it fall back to buffer interface in case rofi is not present.

2 Likes
#7

Even after several years of intensive usage, TeXmacs keeps amazing me. I never noticed the Help->Search->Recent documents (based on docgrep-in-recent and txtgrep in TeXmacs/progs/doc/docgrep.scm). This is great!

It seems only minor changes are required to make this into a more flexible search tool by adding a query to the tmfs-load-handler for grep which allows to specify a search directory.

I’ve created a PR here:
https://codeberg.org/XmacsLabs/mogan/pulls/97

4 Likes
#8

Merged. Awesome work!

I’m considering releasing Mogan Editor v1.1.4 with this new feature in the near future.

3 Likes
#9

Wonder how difficult it would be to integrate the emacs ecosystem into TeXmacs by implementing a plugin that connects to emacs running in server mode?

#10

The feasibility of a plugin depends strongly on which kind of interoperability you would like to have. Which use-cases do you have in mind?

#11

I expect your work to be a very nice contribution to TeXmacs; I think it would deserve to be highlighted.

By the way :wink: it is also possible to create new documents by writing a link to the not-yet-existing document—clicking on the link opens a new document, and one can save the new document, with the filename indicated in the link, after one has written into it.

1 Like
#12

The feasibility of a plugin depends strongly on which kind of interoperability you would like to have. Which use-cases do you have in mind?

First, it would nice if texmacs could become an alternative backend for emacs, much like we already have terminal and pgtk-based versions of emacs.

If that is too difficult to achieve, it would be interesting to explore whether or not a client server model could be developed that would allow one, for example, to edit their org mode files within texmacs by somehow calling out to emacs’ org mode functions from within a texmacs environment over some remote procedure transport layer.

Also wonder if finding a way to leverage edwin (an old MIT scheme-based emacs) or guile-macs in order to create an ‘emacs overlay’ mode that could be activated from within texmacs would be a useful avenue to pursue as well.

Texmacs should seek to recycle as much of the emacs ecosystem as possible instead of trying to reinvent large parts of it.