Journal Search - micropub

Good news: the sharefeed is now an Astro content collection! Thanks to the recent upgrade, it was a lot easier to turn it into a content collection.

Bad news: due to the re-formatting of sharefeed entries I had to do in order for this to work, I might have made your RSS readers see all the entries as new items. Whoops.

More good news: I’ve now got a basic Micropub implementation going that can create, update, and delete journal entries as well as create new sharefeed entries. Using Postman opens in a new tab to test the API, it just works!

More bad news: turns out if I want to make the Micropub implementation work the way I want it to, I need to make sure that both the journal and sharefeed pages server-rendered pages instead of prerendered static pages. I don’t think this will impact performance too much, but if I didn’t do this, just an endpoint on the webserver wouldn’t be enough to automatically update the site.

Eventually, I would want to make this stuff more “isolated”, possibly with a separate Express opens in a new tab worker… but the main concern I have right now is setting up proper OAuth2 Bearer Token authentication. Ideally, I would set up a completely different server for authentication… which costs money. Right now, I just want to get everything else working, then put the project on the back-burner until I get the auth system figured out (since I’ll also need that auth system for the comments section I plan on implementing).

Finally, fun fact: This post was actually written in iA Writer opens in a new tab , transferred to Postman, then sent to my dev server through the Micropub protocol. Yay!

Here’s the obligatory song link opens in a new tab . byyyyeeeeeeeee

A day later and I did the tedious part: converting the big journal page into a series of individual .mdx files in an Astro content collection. That was… fun.

It turns out that Astro slipped an overhaul of their content collection system right under my nose when I wasn’t looking. In their 5.0 update, opens in a new tab they upgraded the system into a full-fledged “content layer” ( heyyyyyy opens in a new tab ) that supports processing all sorts of #content and #data from literally anywhere you want. Part of this update was literally giving you the freedom to process the data however you want, replacing the weird type attribute with a new loader attribute that you can slot either the two built-in functions (representing the two old type’s) ooooooor your own bespoke function for whatever content you want to work with. It’s actually really cool… and slightly breaking compared to the old system. Whoops. So, just to be on the safe side for the future, I migrated from the old system to the new system.

After that, it was pretty simple to get the new journal set up… it was just really tedious to make all the new individual files and copy all of the content over. Honestly, it kinda made me wish I had a generative AI agent handy, but I would rather move slowly and ensure that everything works right than move fast and break everything with AI slop. I also decided to make the now and todo sections of the journal their own files for later use with Micropub opens in a new tab . This way, I can (eventually) not only make new journal entries outside a code editor, but I can also update those journal entries and the now & todo sections outside a code editor!!! How sick would that be???

Now that I’ve done all that, though… look!!! Pages!!!!! It isn’t just one giant page anymore!!!!!! Currently, I’ve got it set to display 10 entries on each page, but I might tweak that later. I’ve also got it set up to only display the now and todo sections on the first page, and only display the really old entries section on the last page. Pretty slick, huh?

My main worry when implementing this was that all of the async stuff I had to add to get this to work would mean excess loading, but I think I can solve that by making this page static instead of server-rendered (the pages are precompiled with all of their content, so when the browser requests the page, all the server needs to do is hand the browser that precompiled page instead of having to go and fetch all of the content it needs first). All of the blog posts are handled this way, so I figured it would be worth it to do it for the journal page as well.

Also, while I haven’t started implementing it yet (I’m waiting for the weekend to do that), I have started looking at the Micropub specification… opens in a new tab and it seems like a really good “baby’s first protocol implementation project”. Everything is very straightforward & well laid-out, and both the h-entry and h-cite microformats seem great for what I am personally looking for. The Micropub protocol itself seems pretty loose when it comes to what you can do through it: as long as the client and the server agree, you can exchange whatever data you want. So, for instance, I can staple a note field to my h-cite entry for use with the sharefeed and hope none of the IndieWeb opens in a new tab people get mad at me for sacrilege. Alternatively, I could make the in-reply-to field in my h-entry its own h-cite object so I use the same base microformat for both journal entires and sharefeed posts, but I think that might be a bit too tryhard… I’ll figure it out once I get a base implementation going for journal entries.

Finally, I made some other miscelaneous changes lately. For example, I got rid of that stupid external link icon and replaced it with a more subtle change: all links (within markdown, at least) now have underlines, and they are dashed if they are internal links and dotted if they are external. Will anyone notice this? Probably not. Do I need a better solution? Yes. But that’s for later me.

also here’s a cool raggae song opens in a new tab kbyyyyeeeeeeee

I think I’m finally starting to get to the point where editing a big long Markdown document for my journal is starting to make less sense. The way I’ve got things set up right now, it’s hard for me to publish my journal entries in my RSS feed, as I would need to scrape this document for date headings. However, if I use Astro’s built-in “content” system instead (like I already do with all of my blog posts), I will be able to parse those MDX files just like I do with my blog posts when it comes to RSS. It will also mean I will finally be able to build pagination into this page very easily: since each journal entry would be its own “object” instead of a part of one massive document, I can make the actual journal page whatever I want and process each entry however I want.

It would also mean I could technically build some sort of API to automatically submit journal entries to my site whenever I want, like through Micropub. opens in a new tab Since MDX is just plain text when you boil it down to its bare elements, it would be trivial to send that plain text through Micropub, save that text to a new .mdx file, then recompile the site to serve that new entry within a few minutes. It would also also mean that I can use Micropub to produce sharefeed entries as well, but that may require wrangling JSON file editing (unless I also times 3 migrate the sharefeed into its own content collection and unify every type of #content on this site under one spaceship roof).

I’ve added my plans to the todo list up top so we can all keep track of what I’m doing… which reminds me, I should probably finish that unlisted article I linked up there, huh? I’m also not sure exactly what I’m supposed to do with the now section… I mostly use it for shitposting or random status updates whenever I touch this page. I get the feeling that I’m supposed to do more with that section, like listing out the books I’m reading (none atm) or what projects I’m working on (this site, trying not to starve)… but I don’t really have much to put in there other than. whatever.

Anyways, I got this indie rock song opens in a new tab that I think I first heard in 2018-ish stuck in my head again so I wanted to share it.

Finally, if you have read this journal entry, you are legally required to email me opens in a new tab with the word “pookie” somewhere in the message. If you do not do so within 24 hours, I will be very cross with you. opens in a new tab Thanks! :3