Org-roam: Rudimentary Roam replica with Org-mode

(github.com)

163 points | by karlicoss 1481 days ago

9 comments

  • ericax 1479 days ago
    I don't use emacs, but I love the concept of organizing your notes and knowledge in plain text, so much that I made my own thing. It's in private beta right now [1], and I've been dogfooding it for the past two months and really enjoying it.

    The graph overview may seem like a gimmick, but it's surprisingly useful in reviewing stale notes. Tracing semantic connection is more fun than just browsing a folder.

    It's also super nice to be able to embed local copies of PDFs, images, and voice notes, so I know as long as I back up my hard drive, I'll never lose them.

    Next I want to add plugins to do periodic reviews of notes, and maybe space repetition for term/definition pairs. Honestly, these efforts only make sense when the data is available for the long haul.

    [1]: https://obsidian.md/

    • bravura 1479 days ago
      So this is interesting. I was ticked off when it says: "what platform do you use the most?"

      I have been looking at a handful of projects in this space, including notion.so and Roam, but am seriously considering moving back to Markdown-folder backed by Dropbox, for which there are a variety of editors including The Archive, and mobile apps too.

      I urge you to allow zero-friction bidirectional sync at the very least. For me, unless I have complete data portability, a note-taking app that might disappear in a year is a non-starter for me.

      [edit: I read deeper and see you have it, thank you]

      • bhl 1479 days ago
        One insight when it comes to building note-taking apps with portability in mind is that when you want to build more rich features, like collaboration or backlinking, is that a database is more suitable to build upon than a basic filesystem. But that architectural choice is at conflicts with portability, because while bringing-your-own-file-system is possible with Google Drive and Dropbox, having your own personal database isn't as popular. Anyone share similar sentiments?
        • slightwinder 1479 days ago
          Yes and no. A database is just a layer. Using a dedicated database and storing your data in it is simple. But nothing prevents you from storing the data (including relevant metadata) outside of it, and use the database just as a caching-layer for it superior abilities in it's own realm.

          Dividing layers is of course more work and has problems of it's own, but works well enough that AFAIK this org-roam here is doing under the hood too.

      • kot-behemoth 1478 days ago
        Have you had a look at Joplin (https://joplinapp.org)? It's mature, open-source, has e2e encryption, is markdown-backed, and supports syncing via Dropbox. Also supports attachments.

        I actually switched to it from org-mode, but still edit the notes in emacs because of keymap (via "Edit in external editor" feature).

      • ericax 1479 days ago
        There's no sync right now, the source of truth is your local folder.

        P.S. When I read bidirectional sync I misread it as bidirectional links, which is something we're working on as well!

    • jcelerier 1479 days ago
      so, what's the difference with Zim which also stores your information on a wiki written with Markdown-ish syntax on your hard drive and is also extensible with plug-ins ? https://zim-wiki.org/
      • ericax 1479 days ago
        I'd say Obsidian is very similar to Zim. Zim is probably the closest thing I looked at before deciding to make my own thing.

        I wanted a few things that Zim didn't offer, such as a quick PDF viewer, bib/reference entries, audio notes (embeddable), and space repetition for term/definition pairs. I realize these needs are highly personal and they will be plugins that you can enable/disable.

        I didn't love the interface but it wasn't a deal breaker.

    • robenkleene 1479 days ago
      Curious why the special `[[wiki style links]]` syntax?

      E.g., why not just use Markdown's existing syntax, like `[A File](a-file.md)`?

      (Cool looking project by the way, all the values are right up my alley.)

      • ericax 1479 days ago
        Thanks!

        There are 3 main reasons:

        1) At least to me, "[Topic](Topic.md)" is the most typical case, where the name of the link is the same as the file name without the ".md", so I feel like not repeating the title twice is better.

        Markdown supports "[](Topic.md)" but IMO the syntax doesn't look as nice.

        2) Easier to distinguish between internal links and external links.

        3) Personally, I think "[[note]]" blends into the sentence better.

        [[wiki style links]] are not as portable as standard Markdown links, I have to admit that. We will support "[A File](a-file.md)" as well for maximum portability, and possibility provide conversion utilities if you decide to move away from the wiki style link syntax.

        • awake 1479 days ago
          curious. how you deal with slugging topics with "Punctuation!". Also do both types of links provide the same affordances? If I write [[Topic]] vs [Topic](topic.md) is there any semantic difference?
          • ericax 1479 days ago
            Not sure I understand the first question, but if the topic itself contains "!", that's fine by the app, as long as it doesn't use the characters that the OS doesn't allow (like slashes).

            The vertical bar syntax is also supported, so you can write [[Topic|display name]] and "display name" will show.

            I usually leave punctuation out of the link, like `[[name]]'s`.

            > Also do both types of links provide the same affordances? If I write [[Topic]] vs [Topic](topic.md) is there any semantic difference?

            No semantic different.

            UX-wise, right now [[ auto-completes from the list of files in your folder, whereas [Topic]( doesn't.

            • awake 1479 days ago
              You answered both my questions! Also saw you created dynalist. Look forward to seeing this new project!
      • patcon 1479 days ago
        Roam does the same thing, so they may have just copied what was already established. I'm sure there's solid rationale on Roam's side fwiw, just don't know it :)
        • ericax 1479 days ago
          Yeah, I wonder why they decided on that as well.

          The first time I learned about this syntax is from MediaWiki's internal link syntax [1], I wonder if there are any predecessors before them.

          [1]: https://www.mediawiki.org/wiki/Help:Links

  • bravura 1479 days ago
    As a vim user, I am not an expert on org-mode. But on the zettlekasten forums, I read an interesting criticism even of Emacs org-mode as forcing a sort of lock-in

    "[Apps like Roam and Notion] force you into a single-app interface with your data. Interestingly, even though it's a plain-text format, org-mode sort of fails here as well because Emacs is the only way to access most of the functionality of an org-formatted file."

    -galen in https://forum.zettelkasten.de/discussion/404/moving-on

    Could someone more familiar with org-mode comment on this?

    • mickael-kerjean 1479 days ago
      There's quite a few other org mode clients:

      - Filestash: https://github.com/mickael-kerjean/filestash => demo: https://demo.filestash.app/login?next=/view/Documents/gettin... (I made this one)

      - Orgzly: https://github.com/orgzly/orgzly-android

      - Organice: https://github.com/200ok-ch/organice

      - Org Web: https://github.com/DanielDe/org-web

      Emacs is undoubtedly the rolls royce but the ecosystem is thriving

      • cam_l 1479 days ago
        Hey, filestash looks amazing. I mean, the org mode client alone looks better than most of the other versions I have tried. Admittedly I don't use Emacs. I have just been looking around for a client for orgzly so I can use it on desktop.

        Will def check it out in full this weekend.

      • indymike 1479 days ago
        There are also several org-mode plugins for VS Code.
        • awake 1479 days ago
          Which one does anyone use seriously. So far they all look like simple experiments.
          • indymike 1478 days ago
            True. For anything other than quick edits... it's back to emacs.
      • noobly 1479 days ago
        There is also Emacs via Termux on Android (it's really not so bad if you enable modifier keys, like volume up for Meta, volume down for Control). This, in addition to Syncthing to sync files across devices, makes for a nice Org workflow.
      • slightwinder 1478 days ago
        Do they support org-babel? Agenda? Org-roam or org-rifle?

        Org-mode is not just a markup-language. The actual value comes from all the tools and business-logic working ontop of the markup. And replicating this quite hard in the short time org-modes fame has spreaded yet.

    • karlicoss 1479 days ago
      I mean, they have a point. I've adopted Emacs mainly because of org-mode, and things you can achieve with org-mode and Emacs are incredible.

      That said, it's just plaintext in regular files on your disk. It can be as complex as you wish. If you use a reasonable subset of org-mode, you can parse it (there are lots of libraries in different programming languages), and then implement any tools you wish :) Not familiar with vim plugins for org-mode specifically, but I've heard Atom support is okay?

      As an example: on Android I'm using orgzly [0] to work with my org-mode files. It shares absolutely no code with Emacs, yet it supports 90% of what I need from org-mode.

      [0] https://github.com/orgzly/orgzly-android

      • minimalist 1479 days ago
        Does their point have any substance? As other people have mentioned downthread, org files are plain text and the functionality of org mode does not rely on information stored in any other database.

        Looking at the comment that raises the objection [0], they advocate for using plain text files and they use a directory of markdown files. If that is the desire, they can use a directory of org files (plaintext like markdown, just different syntax for / italics / and =verbatim= and such.

        Hopefully they're not using any external program to search through those markdown and org files though, else they get forced into an interface to interact with their files.

        One of org-mode's advantages is a broad export support, into everything from even-more-simplified plain text to rich HTML, to everything supported by pandoc.

        [0]: https://forum.zettelkasten.de/discussion/comment/2275/#Comme...

        • karlicoss 1479 days ago
          Yep, in that sense it's really not fair at all to compare it against traditional services that lock you in like Evernote/Notion/Roam.

          I guess I can sympathise in the sense that I really wish org-mode was more widespread outside of Emacs. It feels like a real barrier for adoption (if you're trying to get into semantic org-mode features, not just merely using it for syntax). But I don't see what we can do about it for now.

          • angleofrepose 1479 days ago
            I think rather than trying to get org mode outside of emacs, you want to change the mechanics of entering emacs. In my head this looks like a subset of emacs (in the same way you can imagine notepad a subset of word, or contenteditable a subset of codemirror). If there were an emacs-lite application that just surfaced simple org mode functionality and bindings like moving the cursor or paging removed behind a settings wall, I think adoption of emacs would rise due to the lower barrier of entry.
          • jacobush 1479 days ago
            But I think there is something there. I use org-mode, but even though it is "just" text files, there is no specification to implement against, all the cool stuff Emacs can do with org-mode files.
        • slightwinder 1478 days ago
          Plaintext has no worth by itself. The pwoer comes from what you use the plaintext for. Org-mode has many tools and functionality build around that plaintext which must be replicated outside of emacs when you wanna move away.

          I mean ok, sure, if all you do is maintaining a plaintext-file with a specific syntax and doing everyting else in your head, then the plaintext is good enough, but then you don't need org-mode at all. Then you would be back at the papertrail.

    • rcthompson 1479 days ago
      A good portion of org-mode's benefits come from the way it is integrated into various Emacs features, up to and including the fact that Emacs is completely reprogrammable. You can certainly replicate some aspects of that outside Emacs (and obviously people have done so), but in order to fully replicate that elsewhere, you'd more or less need to reimplement Emacs, or something on a similar scale.

      I think you also have the problem that Emacs' org-mode is effectively the de facto standard with no formal spec, and so every other implementation implements its own preferred subset of that "standard" based on what the author(s) found important.

      (Edit: I could be wrong about more recent non-emacs org-mode implementations. I haven't done a thorough survey recently.)

    • slightwinder 1478 days ago
      Every solution is a lock-in. The only way to be free is to do not note anything at all. But then you are locked into the liberty of nothingness.

      Note-managment on surface is simple, but in the long run in becomes quite complex. At the end it's not all about data, but also how you access them, how the system handles them for you, how you get them presented, etc. There is low-level stuff like syntax of markup languages, and also high level stuff like tooling for displaying and automation of your data. Languages like org-mode or markdown have quite the high level of syntax, with simple things like text-formating, more advances stuff like tables, links, internal links, and high end stuff like codeblocks, metadata-handling.

      And then you have tools like org-agenda or org-babel who work on those features (not sure if markdown has those too), and you build your workflows and life around using those tools, and at some point you are so tangled into an complex environment, that any fundamental change would cripple your abilities you have build over years significantly. And then you are locked-in, even if your data are free and open.

      At the end, your set of tools, the workflows you have build with them, the experience you have aquired for specific domains and the habits you developed over the years are your cage that prevent you from moving on to different systems.

      Org-mode has many such features, tools and habits you imprison you. There is org-agenda, org-babel, internal links, headline-tags, todo-tags(?), but also the outline itself. Those are the usual popular tools and features, and they are not much used in other systems, and hard to convert when you aim to move out.

    • rasengan0 1479 days ago
      I used to be a Org-mode evil user and still have my ~/.deft accessible as a VimWiki directory. They are all just plain text and accessible from vim or emacs or any other plain text editor so I do not see any lock in.

      While VimWiki is great with backlinks and would have better adoption with a rename to the current hotness: VimRoam or VimWhateverSaasNotetaker, linking is trumped by Ripgrep/Fzf/Ag/Ack functionality.

      I think the take home is stated in LONGEVITY of https://org-roam.readthedocs.io/en/latest/

      • sooheon 1478 days ago
        I'm using Roam for now with the assumption that a json dump would be straightforwardly convertible to a filesystem of .md or .org files. These services usually have full-text search as well (Roam does).
    • MikusR 1479 days ago
      Outside of Emacs org-mode files have the same functionality as markdown files, but with a slightly different syntax. https://news.ycombinator.com/item?id=15321850 https://news.ycombinator.com/item?id=19622019
    • SJetKaran 1478 days ago
      True, to use the full functionality of org-mode, you do need Emacs. As long as one uses a note-taking system that uses plain text, they are good. Saying that, org-mode has a specific syntax that all org files adhere to, so that you can re-format the files to any other format whenever you want (or use the in-built export options in Emacs/Org-mode to convert to Markdown/PDF/etc).
    • Torwald 1479 days ago
      There is also web org at https://org-web.org

      which means that now

      > Emacs is the only way to access most of the functionality of an org-formatted file

      is wrong.

      • slightwinder 1478 days ago
        >> Emacs is the only way to access most of the functionality of an org-formatted file

        > is wrong.

        Depends on how you define "most of the functionality". Org-mode is not just the syntax of org-files alone. It's mostly the tools generating and using the content from org-files.

        • Torwald 1477 days ago
          > It's mostly the tools generating and using the content from org-files.

          What do you mean by that?

          I would say the heart of org-mode is the org-agenda views, or is it?

    • Torwald 1479 days ago
      I used to do TONS of stuff in org-mode, I lived there as they say, for years. I moved on. My current text editor handles org files just well.
    • bachmeier 1479 days ago
      Why not export to a different format like html or markdown if you give up on org-mode? Edit: Notion lets you export to markdown.
      • bravura 1479 days ago
        This is, in fact, the approach the poster I link advocates for. So I am most interested in note-taking apps where you can swap the editing UX seamlessly, and not be locked into a particularly non-portable format.

        '"So the question is: How important has it become for anyone to really have plain text file system based zettels? Has it ever been a real limitation to store zettels in a richer format in a more proprietary store like Evernote?"

        My answers would be "very important" and a resounding "yes". The value of simply formatted plain-text is not in future-proofing data — pretty much any program can export my data in some reasonable format these days. The real benefit is that, because there is no difference between the internal and external representation of plain-text data, I am free to use whatever program is most suitable for my needs at the moment. I add new zettels to my zettelkasten mostly through The Archive, but I also have my zettelkasten added as a folder in FS Notes (where I keep my short-lived project-specific notes as well as my non-zettel reference notes) and in iA Writer (where I do my long-form composing). If I'm putting a table into a file, I'll open it up in Vim and use table mode. Sometimes I'll take live notes in org-mode, and I'll want to look something up so I'll pull up my archive in deft. Tweaks to the ZK can be done directly in any of these programs with no context switching. I find it immensely valuable to have my archive available directly in these programs. And no waiting for app authors to pretty please implement some feature.

        I've found tremendous benefit from just deciding to use a directory of markdown files so I can stop wasting energy finding the "perfect" solution. For my usage, the union of functionality of the gaggle of apps that operate on a folder of markdown files is more than enough.'

    • amiga_500 1479 days ago
      This is a really odd criticism that I don't think I could have dreamed up, had I set out to.

      It's plain text, any other app could parse it if they added such a "mode" and you can export to other formats.

      Surely this kind of criticism could be leveled at countless applications, and many wouldn't have the export retort as a defense.

    • lallysingh 1479 days ago
      Org has plenty of export options.. and emacs is pretty scriptable.
  • dmortin 1479 days ago
    The graph seems more like a gimmick, but the automatic backlinks are nice.

    Org mode is a really good text-file based solution for notes, especially with tools like org-rifle: https://github.com/alphapapa/org-rifle

    The other day people wrote in an other thread they use grep for their text notes. But what if you have notes with longer, multiline texts? With org-rifle the input words can be anywhere in the multiline note and they can be in any order.

    • karlicoss 1479 days ago
      I've got a massive org-mode database, and in 90% of cases I'm using ripgrep (helm-rg) to search over it. In addition it searches in chat logs and other non-org mode files. Remaining 10% of searches is probably org-ql (e.g. tag search). Tried org-rifle too, but also wanted searching within the contents, so ended up using ripgrep.

      I describe my setup here: https://beepb00p.xyz/pkm-search.html#other

    • secstate 1479 days ago
      I'll say that I've been using org-mode and it's agenda to manage todos and take notes for a long time, and Roam is whole other animal. The problem I've had with Rifle is that words can mean different things in different contexts. Rifle finds every time I used the word "depression" regardless of how I used it. Roam gives you any easy way to add just a little extra symantics to your notes that dramatically increases the signal in the backlinks, and astronomically improves the graph view.
      • dmortin 1479 days ago
        Could you show an example of those "little extra symantics" which helps you with this?
        • secstate 1479 days ago
          For sure!

          So if I'm reading an article about the fall of the Roman Empire, and I learn about how modern scholarship does not talk in terms like "fall" much anymore, but rather civilization transformation, I might make a new stub article titled `civilization` and then another one for `civilization transformation`

          Now, if I read a story about Asoka's Empire in what is now India, and I read about how it underwent a collapse, I might tag it with `civilization transformation` as well.

          Another story about the nature of modern civilization and the travails of people who treat their kids as a personal project to "build" as opposed to a living being to nurture and encourage, I would tag that with `civilization` but not `civilization transformation`

          Ripgrep'ing my org files for "civilization" would turn up all three of these backlinks. Grepping does not understand context. But I understand context. While `civilization` is part of `civilization transformation`, I don't want `civilization transformation` surfaced for every search of `civilization`

          This is a rather contrived example with some obvious issues, but I suppose a good analogy would be that you can drag fish for scallops and get every other living thing off the bottom in your net, increasing the work to clean it out. Or you can dive for individual scallops and take longer but have less work to do when you surface. Both methods have merit, but one provides a much cleaner signal.

          Practically speaking, I USE my Roam note's backlinks. While I could grep my previous notes, in practice I never did because it was too much work.

          • b3n 1478 days ago
            If you are using backlinks, why not just grep for `[civilization]]` or `[[file:civilization.org]`?

            I actually have a binding in Emacs to run ripgrep on all files in the current directory, which have a link to the current file. It's very simple, and runs in a split second even with 1000s of files.

          • dmortin 1479 days ago
            Thanks. I was curious why org-rifle is not sufficient for search, but you just describe searching by tags which org-rifle can also do, or searching within a tag.
    • jarvist 1479 days ago
      grep -B / -A / -C [lines Before / lines After / lines of Context] is very useful for when you have multi-line text you want to extract.
      • dmortin 1479 days ago
        The point is searching multiple terms any order in a note. With org-rifle I don't have to think about the number of context lines or anything and I just type the terms and it finds the notes which have all of those.
  • sooheon 1479 days ago
    Set up org-roam for evaluation, but when it came time to actually do research, the thought that has already gone into Roam UX won me over. Everything is theoretically replicable in elisp, and there are key bind related bugs in Roam that would be an instant fix in emacs, but "good enough and works right now" beats "could be perfect with weeks of fiddling".

    Standout features: referencing ideas is zero friction, whether they exist or not. Universal search works. Editing note titles fixes all references. Days being a first class citizen, where daily prompts thread your notes together and you can reference future dates to have the backlinks show up on that day, is genius.

    • laughingman2 1479 days ago
      Same thing happened to me. I was using org-mode for GTD and note-taking for the past six months. I shifted all my notes to org-roam with links between them. But then I tried roam, it's UX really won over me.

      One thing was accessing my notes with all roam features in all the devices. I don't have to setup dropbox/syncthing/whatever in every machine I own and install emacs with my settings every time. And mainly mobile, If I use orgzly, I don't get any benefit of org-roam in it. So I gave up and embraced roam cloud.

      I think there is a huge gap to be filled for a device agnostic emacs that works well on mobile (touch), browser, native and is programmable.

  • chrisMyzel 1479 days ago
    Great - i started yesterday using org-roam and got many ideas out of my head. Found it by reading about zettelkasten (in the fabolous "Where good ideas come from - The History of innovation") and DevonThink (which i really miss, not avail for linux) and googled for alternatives
  • ElDji 1479 days ago
    Such kind of text base note wiki with graph showing links can be done use tiddlywiki and http://tiddlymap.org/ addon.

    Works like a charm, open source, self hosted

  • whitten 1479 days ago
    There was some mention of using Org-roam as an editor in a wiki. What is the process to attach it to the Media-wiki software (used by Wikipedia, etc)

    Sorry if it was on the github site and I missed it.

    Thanks for the presentation, it looks interesting.

  • elcomet 1479 days ago
    Does someone have a good tutorial to learn this and emacs? I'd like to try (I tried roam research online but couldn't connect, and I'd like to learn also about emacs).
    • karlicoss 1479 days ago
      I'd say, first of all, use an existing Emacs setup, like Doom [0]. It's got lots of very reasonable defaults, so you will spend less time being mad at Elisp and more time doing something productive. In addition, if you are used to Vim, you won't have to relearn keybindings.

      I personally haven't got to try org-roam yet (I use a somewhat similar workflow though). In terms of learing org-mode, I recommend trying org-capture to quickly write down your thoughts, tasks, etc, and then seeing where it goes from there. I started with just an org-mode file with pile of notes and I gradually discovered the features when I needed them.

      [0] https://github.com/hlissner/doom-emacs

      • rasengan0 1479 days ago
        this doom-emac is wonderrful thank you for the reference!
    • yewenjie 1479 days ago
      There are many many tutorials out there in the internet for learning Emacs. I would agree with OP's suggestion to start with a starter-kit emacs, like Doom or Spacemacs.

      You can check out this video which introduces org-roam for total newcomers to the Emacs ecosystem.

      https://www.youtube.com/watch?v=Lg61ocfxk3c

      PS: The installation for org-roam in doom has gotten significantly simpler since this video. Check out the official docs for updated instructions.

  • mateuszf 1479 days ago
    Is it possible to link to a specific header? What if I change the header name?
    • yewenjie 1479 days ago
      AFAIK the org-roam main author has strong reasons for not allowing links to headers. There is another project that you can check out - org-sanpo [1].

      1. https://github.com/kamoii/emacs-org-sanpo

    • dmortin 1479 days ago
      You can assign ids to headers and link to that.