I'd greatly appreciate it if you could share your thoughts on the following: 1) List Organization: How do you manage your reading, watching, and listening lists? 2) Content Capture: How frequently do you find yourself copying quotes or highlights from articles/books/podcasts/videos? 3) Information Retrieval: How often do you refer back to your saved highlights? 4) Knowledge Storage: What methods do you use to store professional knowledge? 5) Challenges: What do you find most challenging about managing your knowledge base?
I’ve been caching bits and pieces of information I’ve found online since the late 1980s and anything more sophisticated than plain text has gotten me burned. Anything proprietary is guaranteed to suffer from business development needs pushing for incompatible upgrades to an undocumented format followed by an “our incredible journey” termination of service.
Have learned the hard way that you cannot rely on anything staying online long term, even without organizational change or dissolution. The needs of the business change and one day the entire archive you've been relying on for reference has been removed.
I got into the habit about a decade ago of using Evernote as a stash for items like web pages only to find its web page snapshots frequently looked like they grabbed the content of a page only to lose it some time later. So, back to stashing items in a filesystem.
So…if it’s something I find useful I grab it and stash it in my laptop filesystem (which I back up to my own split geo NAS setup).
I don't know how you build a product to satisfy someone like me because you end up having to fight the distrust that's built up over decades, no fault of your own.
I also agree that if it's an online-only resource, then it's not preserved at all.
Directly copying them? Almost never - at the very least I'll rewrite or rephrase them to assist in committing them to memory.
The lower the bar to "saving" things, the less impact they'll have on you going forward. Think the invention of the digital camera, and the thousands upon thousands of largely forgettable pictures.
What do you find most challenging about managing your knowledge base?
Until I wrote a personal RAG assisted LLM (Mistral based) chat system, it was difficult to retrieve multi-note information in a way that didn't involve searching for each individual note. I have around 10,000 "rich markdown files" (e.g. contains links to media) representing two decades of notetaking for personal projects, work, etc.
Tagging goes a long way but Context-Aware notes aren't really a thing in most note taking applications.
Final Thoughts:
The tools/applications/integrations (GetPocket, Joplin, Logseq) have never made it easier to take notes.
Reading these forums for years and seeing the same reoccurring topic (best note taking tool, how to take notes, etc) I also think that there are three types of people:
1. People who take notes
2. People who don't take notes
3. People who like to discuss the theory of note taking
Now that djvu JBIG2 patents expired I jotted a script from parts to convert djvu to pdfs much better than earlier alternatives (with converted file size swolled a typical mild 66% more than djvu). I read pdfs with sumatra that has customizable external "viewers", that means invoking stuff on the pdf page with alt_f + key. One "viewer" copies a link to the current page in the clipboard. Now sumatra supports pdf annotations, made a couple scripts to convert annotations to pdf clickable hyperlinks (and other to undo that). So basically I have a pdf folder and a markdown folder (managed via Obsidian, if you ask) all interlinkable. I read, and copy and paste into markdown, and paste also a link to return to the source from the note. Pay mathpix suscription, there starts to be free would-be replacements. Use Mozilla reader mode chrome extension and print to pdf to save neat pdf versions of blog posts. Finally I have an e-ink e-reader (2nd hand sony dpt-rp1) and am just writing scripts to sync it with windows via an always-on NAS (the e-reader registers itself via zeroconf, and in the NAS via python one can hook a listener that will call unison two-way sync tool). Add also Omnivore.
Then I get my own knowledge database that I can query with natural language.
I want to be able to ask my second brain "what do I know about x?" And retrieve a list of articles I've read on the topic, conversations where I've talked about it, projects I've used that concept in. My meat brain doesn't have that kind of recall, it takes a lot of mental effort to sort through the enormous pile of things that I know.
The less effort I have to spend on organizing and maintaining my knowledge, the more effort I have available to use it.
It seems like this ideal is just a few years away. Several people are already building systems like this, but it doesn't seem that anyone's actually produced a polished product yet. And absolutely not one that is local-only.
This is not the type of thing that anyone should ever trust to "the cloud". This is something that must be run locally on hardware you own and control.
When you say "Several people are already building systems like this", could you please name a few? What feature do they lack from your point of view?
About trusting the cloud, what do you think? If the service would offer both local storage and secure synchronization in the cloud, would it be acceptable?
As to cloud, never. This system would ideally contain every detail about my personal life, and frankly I think you'd be insane to trust google or OpenAI with that no matter how heavily you encrypt it. This is a whole new order of magnitude in personal, private data. I wouldn't trust anyone at all other than myself with this.
My server at home gets free solar power and free geothermal cooling. I have no reason at all to ever want to use someone else's computer. I have plenty.
Someday I will build my own clone of Simplenote/Notational Velocity/Taskpaper/Drafts/Bkmks.com...
1. From mobile browser, share to Simplenote adds a new note with title + url.
2. Not often.
3. Not often.
4. Also Simplenote. Also used Notational Velocity: one note per JIRA ticket.
5. I wish Simplenote's tagging system were better (more like TaskPaper)
- I wish it were simple to add (markdown) images like Github. (CTRL-V automatically uploads and inserts markdown).
- Also Simplenote added some features that get in the way (I just want to type `[`, but Simple note opens this inter-note linking interface)
- Notational Velocity supported rich text, but I just wanted everything in plaintext, especially when pasting information into a note.
Do you feel comfortable to use two apps (Obsidian and Notion) simultaneously? How do you choose which content goes where?
5) Did you tried tagging? If yes, why it does not work for you?
Yes, I feel comfortable, it's a matter of making the habit. Everything specifically related to the knowledge of my profession I use Obsidian. For everything else I use Notion. As for the use of tags, yes I use them.
Best,
Tried a couple other options and decided flat text is the only thing I’ve got long term confidence in
Briefly ran an obsidian setup on top Of that but leaning more towards simple folder structure.
Expecting that to play well with future LLM tech too. Flat markdown text files should ingest well
Data aggregation? Summarisation? Retrieval? Or maybe some ingestion post-processing?
I'm getting ideas of making a service that works as a git origin that you can just push to update and then access in a web UI to get nice features, maybe a project for a weekend.
I just want the data to be in right format once o eventually get round to it
2) Zero times. I do use ArchiveBox for snapshotting interesting articles and searching for them later, but mostly out of fear of some things falling off the Internet (which is increasingly common)
3) In practice, very very seldom. I stopped using delicious and Instapaper and whatever else because search is a solved problem.
4) Mountains of documents and plaintext notes, with file names prefixed with YYMMDD. Seriously, I can track down what I was doing ten years ago in under 5 seconds.
5) The fact that people keep reinventing note-taking apps and none of them do what I need, so I stick with my wiki (taoofmac.com) and my filesystem + Spotlight.
2. Hmm probably never?
3. Probably never as well?
4. I've tried obsidian, regular plain text files with vim, notion, nothing works for me. Now I just use bookmarks with webtag.io since it's quick. I don't really write notes, just store references.
5. I can never write store and manage any knowledge-base, my brain seems to just not work that way. I find it too easy to just delete things I don't like, until I realise there's nothing left. I'm too perfectionist, to the point that I end up with nothing time and time again.
What that looks like is that I use LogSeq's daily journals and extensive use of tags and properties. So maybe that looks like (simplified example):
(for list organization)
LATER #towatch “20 Days in Mariupol” #documentary - Sandra's recommendation category:: film
Then, I might have another page with a query like:
{{query (and (property :category "film") (todo later now))}}
but I can also go to the tag page documentary or towatch and have anything I've tagged there. Now, I don't tend to get too deep into watching lists, and stick mostly to non-fiction.
I have a book template and article template that focuses me on key insights, and I just take general notes on what is surprising to me or useful to go back to. I take liberal copies everywhere and clean up later. I don't go back daily, but I find it helps when I want to quickly brush up on something or if I want to refer to it in a way that I can send someone else something. ___
Now, to your 4) most of my lists are more work-focused. Again, I use PARA as my storage basics and I've adjusted as I've went. I blur projects and areas more than Tiago does, and use intermediate documents a lot less. I also use daily journals as a lot of my organization because my temporal memory and search generally lets me remember where to put things, but it does mean I have to get back to people later. The real problem is managing the knowledge base and I just keep everything in daily journal until I need to organize it.
LogSeq does generally use markdown and lists, but the more features you use the further away it becomes useful outside LogSeq. If I ever moved away from it, my search would largely be "grep".
I never manage my reading. Plex manages my watching/listening. I almost never summarize things that I find online and almost never refer back to those summaries. I use a notebook to store knowledge. The most challenging part of managing the notebook is that it takes up limited desk space.
Speaking candidly:
I've tried a number of systems such a Dendron, Joplin, Obsidian, etc. and nothing feels as good as pen and paper. In theory you'd think these systems are fantastic but I've come to the conclusion that taking notes is more than the raw information taken down. With a notebook I feel like I have a mental map in my head of the geography of the notebook, the places different information is stored, how the notebook feels. It's also low friction. At any time I can touch a pen to paper and nothing can stop me.
Over the years I have tried a few different tools but didn't get along with Notion etc - I get bored easily and don't want to spend days setting up a system as I view that as procrastination.
2) Every day. Whenever I find something worth copy/pasting.
3) Only once in a while. But when I read back on my saved stuff, I have a lot of fun.
4) Copy and Paste to a rtf file, or anywhere I can paste text and images and add some formatting. Has to be simple, quick and easy to access. MS Wordpad is the best ever, and I'm not a MS Windows fan. Make sure to put a good title and include tag words that can be searched later.
5) That I can't find anything better than MS Wordpad. It has an I-don't-know-what kind of simplicity that makes it a pleasure to use. And I'm not a MS Windows fan.
2/ rare. I have a bad memory. I don’t even know that I don’t know.
3/ 1-5% of the time I reference a pocket link
4/ Anki, google docs, my blog.
5/ writing takes a lot of time. Even with ChatGPT help
1. what is the hardest part using Pocket? 1.1 how often do you open pocket to read something? (just approximately)
1.2 Just to clarify: do you keep your Pocket list clean? Or you are using it like a reference list for future?
2. Have you ever caught yourself on though that you can’t find something you read previously? 2.1 if yes, how often? 4. Just to clarify: Do you use anki cards to learn better?
5. could you tell me please about the last time you wrote something?
1A/ if pocket can extract the text into a reader view, it will cache it for offline viewing. If pocket can’t, then I don’t get to read the article I was expecting to read on the airplane.
1B/ 20-40% of the articles I listen to while walking or driving, but they only work if the reader view is available, I must be connected to the internet, and the voice sounds pre-2022.
1C/ the search function isn’t great. I think if I pay for it, it will do full text search. But I think it’s dumb that I have to pay for ctrl+f.
1.2 not sure what you mean by clean. It’s not well organized. I have like 5 tags I use around topics that might be useful later, but I rarely archive articles or properly tag them.
2/ it definitely happens maybe twice per year. I use Anki occasionally, but only for deep topics (learning a new programming language, leet code), and not to learn blog content
5/ I write code and slack messages daily. I probably write internal documents one to four times per month. I’m trying to write one blog article per week, but it’s more like one month.
I don't keep any such lists, aside from bookmarks which I keep on a bookmark server that I run.
> 2) Content Capture
Extremely rarely. If I come across something I want to save, I prefer to rewrite it in my own words because that increases my understanding and retention.
> 4) Knowledge Storage
I keep everything (except bookmarks) on a wiki server I run. This includes specialized knowledge, notes about projects and research, manuals, datasheets, etc.
> 5) Challenges
Honestly, I don't think of anything about it as challenging, but if I had to pick something, it's probably the mechanics of using the wiki. But my process is one I've honed over decades and it serves me extremely well.
Ipynb content is mostly static content I'd like to share w people. It requires a git push to update.
.txt notes are stored on a php based shared host and managed via a hidden url. I use it for life notes, and todos. It's been working great so I plan on using this approach to add a blog to my site at some point.
My downloads folder just piles up till I sort what I can and dump the rest in my archive. The archive is real and an llm will sort it all out for me one day :) I hope.
Tons of other successful people in the history have used the notecard system to create a commonplace book.
What I'm not a fan of is people spending hours on setting up these productivity systems
Obsidian for storing raw information and connections, and Notion for my to-dos/trips/lists.
2) Any time I share an article with friends and want to emphasize a snippet.
3) I usually just Google whatever source I need at the moment.
4) Brain.
5) Too many ideas come up and I don’t end up writing them all down.
2. All the times, typically during the conversation
3. Any time I want to dig up with my old crap
4. A lot of HDDs
5. Backups: how to not lost the info because of either tremoring hands or the war which is going on.
1. Could you tell me the last time you faced this? 1.1 What was the most challenging part?
2. How often do you face the issue when you can’t find something you read/watched/heard? 2.1 Do you feel frustration when you can’t?
3. How often it occurs? just approximately
Stay safe, I know what you’re feeling
2. Always! 2.1 I know I can write down something belonging to one of important projects of mine (writing a thoughtful blogpost, collecting samples for future music track, grabbing something from the Internets for the sake of archiving) and if it is important but I have not done the simple thing of growing the work I feel like very frustrated. Any other cases of forgetting are not important for me, but I can always start a new project if I am sure I want even more information grinding in my life.
3. I have a rule of three slots. If I have remembered about some issue once a week, it is the 1st slot. Next time I move the issue to the 2nd slot. Third time is the time I got to solve the issue. I think I have maybe 4 unsolvalble issues per year which are about mismanaging the data issues - typically ends up in reorganizing something in my life, maybe buying something or writing a long letter to someone.
Thank you. But I do not feel like I understood your intent of asking.
2) Obsidian web clipper[0] and a Keyboard Maestro powered hotkey that opens up either my daily note with two taps or creates a new Second Brain note with three
3) When I need it. I'm pretty good at keeping an index in my head of stuff I've seen "sometime somewhere". By having everything in Obsidian I don't need to start searching browser/search engine histories to figure out what was that one site with that bit of knowledge.
4) Obsidian Second Brain. When I spend too much time (by feel) solving something, I spend a few minutes writing the solution down along with links to the sources I used. The solution is generalised so that it's not infringing on any employer copyrights.
5) Consistency and not bikeshedding about knowledge base systems too much.
I've used Evernote (one of the OG Mythbusters recommended it like 15 years ago or something). The OCR system was a life-saver in a few occasions. Then they did some stupid Silicon Valley crap and I moved between OneNote (amazing if you have a laptop with a stylus), Emacs (org-mode bikeshedding took way too much time), Joplin (open source, but the storage format is meh), Notion (pretty and the DB stuff tickles my nerd brain the right way, but too slow for my needs) and maybe a few others.
Now I've been using Obsidian daily for 6 months and even finally bought their Sync service, which works super fast - the free ones aren't bad and iCloud syncing works well too, but the official one is just that bit better. In the end it's just a bunch of files in a directory structure so I don't have to worry about vendor lock-in. Replacements Omnisearch, Templater and QuickAdd are the only essential plugins I need if I want to move to another platform.
[0] https://stephango.com/obsidian-web-clipper