Site Updates: /listens and /photos pages

Inspired by Jonathan LaCour's January post about tracking his podcast listening habits, I recently decided to bring in my own listening history into my website.

Like Jonathan, I use Overcast to listen to podcasts on my phone, and I was surprised and delighted to learn that Overcast has an "All data" export option on the accounts page that includes per-episode details such as whether you listened to an ep, and when you last interacted with it!

I took a look at Jonathan's script to automatically log in and import his Overcast data and figured I could make something a little more minimal that would work with my Hugo site.

My process follows three steps: download the overcast.opml full export file, run a script to generate the Hugo posts, and run a script to grab extra podcast metadata like cover art.

You can find my scripts in this gist if you're interested in learning more, but they may not be that useful for anyone without my exact Hugo config.

You may notice in the scripts that I have a "block list" of podcast URLs I can add. That's because I wanted my listen posts to helpfully link to and embed audio whenever possible. However, some of the feeds I subscribe to contain private or for-pay content - particularly a couple of podcasts I support on Patreon. I do want to track my listening habits for these podcasts, so I will likely start handling them differently by leaving out the embedded audio and linking directly to Patreon pages where possible.

Running the script, I was able to happily show everything I listened to as just another type of post on my site. However, it turns out I had (and keep making) a lot of listen posts. Visually they were drowning out all my other activity on my site.

So I have sequestered them on a new Listens page where you can only follow my podcast consumption history, keeping it out of my main feeds.

Similarly, I have been interested in having a nice Instagram-like page that shows off my photos in a grid. So I made a Photos page to show off those posts.

Both of these pages have microformats2 feed information embedded, so if you have an Indie reader, you might be interested in following just those pages!

#listen #read #IndieWeb #site-update

IndieWebCamp Austin is this weekend! You can still register here.

@ag_dubs @steveklabnik Congratulations! Austin’s great.

Coincidentally, I’ll be there this weekend @IndieWebCamp Austin, hosted by @manton2(@microdotblog) https://2019.indieweb.org/austin

Got plans? You should join us!
Would be great to hang and hack on things together.

And pretty sure Manton is getting breakfast tacos.
@IdentityWoman @csageland @generativist
join me(@t) & @Johannes_Ernst at this Wednesday’s #IndieWeb Meetup:
17:30 Homebrew Website Club #SF @MozSF
RSVP: tantek.com/2019/051/e1/homebrew-website-club-sf
Wiki: https://indieweb.org/events/2019-02-20-homebrew-website-club

We’re into humans-first & UX>protocols.

Homebrew Website Club SF!

When: Where: Mozilla San Francisco Host: Tantek Çelik

17:30: Optional writing hour and quiet socializing
18:30: IndieWeb demos and hack night!

Homebrew Website Club retro 1980s-style logo

Topics for this week:

Join a community with like-minded interests. Bring friends that want a personal site, or are interested in a healthy, independent web!

Any questions? Ask in #indieweb Slack or IRC

More information: IndieWeb Wiki Event Page

RSVP: post an indie RSVP on your own site!

The Web Through a 30 Year Old Lens

A team of people, including Jeremy Keith whose writings are part of my daily RSS infodiet, have been doing some awesome web archeology. Over the course of 5 days at CERN, they recreated the browser experience as it was 30 years ago with the (fully text based) WorldWideWeb application for the NeXT computer

Hypertext’s root, the CERN page in 1989

This is the type of pages I visited before inline images were possible.
The cool bit is it allows you to see your own site as it would have looked 30 years ago. (Go to Document, then Open from full document reference and fill in your url) My site looks pretty well, which is not surprising as it is very text centered anyway.

Hypertexting this blog like it’s 1989

Maybe somewhat less obvious, but of key importance to me in the context of my own information strategies and workflows, as well as in the dynamics of the current IndieWeb efforts is that this is not just a way to view a site, but you can also edit the page directly in the same window. (See the sentence in all capitals in the image below.)

Read and write, the original premise of the WWW

Hypertext wasn’t meant as viewing-only, but as an interactive way of linking together documents you were actively working on. Closest come current wiki’s. But for instance I also use Tinderbox, a hypertext mindmapping, outlining and writing tool for Mac, that incorporates this principle of linked documents and other elements that can be changed as you go along. This seamless flow between reading and writing is something I feel we need very much for effective information strategies. It is present in the Mother of all Demos, it is present in the current thinking of Aaron Parecki about his Social Reader, and it is a key element in this 30 year old browser.

Once again, my backpack is heavier than my roller bag. This is what happens when I carry livestreaming gear to #IndieWebCamp Austin. #travel
#IndieWebCamp #travel #indiewebcamp

Wat is een #indiewebcamp ? Frank Meeuwsen, legt onze verwachtingen voor #indieweb camp Utrecht (#iwcutrecht) uit. Ben jij er bij op 18 en 19 mei? https://diggingthedigital.com/wat-is-indiewebcamp/

The things we do to read 💩 emoji

Ok I totally stole the title from this article, which was really good and help me fix my problems so I wanted to link to it.

I recently upgraded all my servers from Jessie to Stretch, which was long overdue. The catalyst being that certbot from Let's Encrypt started complaining about the security I was using to fetch new certificates. (It complained in a very nice way and helped fix the problem, I should add.)

Anyway the upgrades went fine, Debian is very good at that. The problem was that part of the upgrade was switching from MySQL 5 to MariaDB, and all my tables were using an old character set and collation type. This showed up as emoji not rendering properly in my reader. Easy fix: switch the reader items table to the utf8mb4 character set and collation type to utf8mb4_unicode_ci. Problem solved.

This is where the fun really started though, and where my desire to understand the inner workings of databases started fading... My reader became so slow at loading items that it was unusable. The rest of the site worked fine, so I isolated the problem down to one database query. Due to the way I now have channels set up, this was a particularly complicated query running on the largest table in my database, reader_items which was 400k rows and growing. I delved deeper into the murky world of database performance, learning about sargable queries and how to keep your indexes fast.

Making sure all the tables joined in the same query have a matching collation type seemed to do the trick. My query which had blown out to over a minute was now running in a couple of seconds. A good reminder to look after your indexes! Since I was well and truly into database tuning now though, I decided I could get more performance improvements out of my reader.

The problem with adding new feed items to the same table is that you generally only want to read the new stuff. It's great to have fast indexes, but most of the time I really just need smaller tables. To do that my reader needed to be able to look up items in an arbitrary number of tables, but optimised to find new items in the first table it reads from. It can now do that, and the data partitioning process is automated to keep the items tables small. It can also handle reading across tables to return the correct number of items requested.

The problematic query now returns in less than a second for the optimised case. (I would measure it but it's not a noticeable part of using the reader any more.) My plan to fix the performance issues before making this change was to increase the specs of the server it's running on, which are pretty modest for all the work it's doing. I do however like the idea of improving the efficiency of code rather than throwing more hardware at a poorly running solution.
Throwback to a rainy and grey New York last week when I lost my voice but powered through my talks anyway. Here's to another packed week! Three talks in two cities followed by #IndieWebCamp Austin!
#streetcapture #streetphoto #streetweekly #urbanaisle #rawurbanshots #streetgrammers #streetsofnewyork #pmwintergridchallenge #nyc #newyork
#IndieWebCamp #streetcapture #streetphoto #streetweekly #urbanaisle #rawurbanshots #streetgrammers #streetsofnewyork #pmwintergridchallenge #nyc #newyork #indiewebcamp

Week Notes 19#07

It looked like a pretty regular week, but then I developed a fever, so had to stay at home some of the days.

  • Had conversations with the Open State Foundation deputy director and the one remaining board member I hand’t met before yet, Noor
  • Discussed with Frank Meeuwsen when and how to organise an IndieWebCamp in Utrecht this spring
  • Worked on open data for a province
  • Caught up with André Golliez who has started a new company with two partners in Switzerland, and we discussed their new propositions and services
  • Started planning a stakeholder mapping exercise for a client
  • Had in-depth conversations with my The Green Land partners about our plans and course for the coming year
  • Spent Saturday in Enschede visiting our old neighbours for the birthday of their sons, and then an informal wedding reception at Henk and Melina’s, meeting many people we hadn’t met for a while. Our little one had a great time, so she didn’t mind we stayed a bit longer to catch up with people.

Just posted the new Core Intuition. More about WWDC travel, IndieWebCamp Austin, and Daniel considering dropping Blogger from MarsEdit.

#IndieWebCamp #Austin is next weekend!
Get a #website, #ownyourdata, cut back #socialmedia, or hack on #dweb #webstandards.
Join @manton2(@microdotblog) @aaronpk(#indieauth #webmention) @dshanske(#WordPress #IndieWeb) & more!

RSVP: https://2019.indieweb.org/austin
#IndieWebCamp #Austin #website #ownyourdata #socialmedia #dweb #webstandards #webmention #IndieWeb

IndieWebCamp Austin is coming up in a week at Capital Factory. Bunch of IndieWeb-related things I want to think about for Micro.blog that weekend. Everyone’s welcome!

Awesome! Let's try to get you on there! So I think your issue is that your authorization endpoint is set to indiecert.net which doesn't seem to exist. You might want to use indieauth.com instead?

Pixellated Microformats SVG button to celebrate the 15th anniversary of Microformats being shown off at ETCON 2004 by Tantek and myself.

15 years ago @KevinMarks & I introduced #microformats @oreillymedia ETech 2004 “real world semantics” session, before #RDFa #microdata etc., with <15k #HTMLFirst + inline #CSS+#JS: tantek.com/presentations/2004etech/realworldsemanticspres.html (faster than JS-only slides)

Previously: tantek.com/t4UY2 (https://twitter.com/t/status/433494367601717248)

Since then:
* drop the (X), just HTML
* meta tags still web alchemy, even more bloated DRY rot
* h-entry superceded hAtom/hentry/rel="bookmark" - microformats.org/wiki/h-entry
* recommendations: hReview still more reliable than anything from Google (https://aaronparecki.com/2016/12/17/8/owning-my-reviews). Use h-review today microformats.org/wiki/h-review
* syndication: h-feed publishing/consuming growing rapidly. microformats.org/wiki/h-feed
* music playlists still unsolved for any cross-site/service use-cases

Blogged 15 years ago:
* tantek.com/log/2004/02.html#d12t0012
#microformats #RDFa #microdata #HTMLFirst #CSS
Hello from Homebrew Website Club NYC! #indieweb
I'm hosting a Virtual IndieWeb Meetup next week. Want to talk about things you've recently done with your website or brainstorm ideas about what you want to do next with it. Come join us for a friendly night about your presence on the internet.
Replied to
20190209_144138

Trying to see how this ends up on my blog and perhaps Flickr, when replying using Yarns microsub server in combination with Monocle reader. Typing this directly in my reader.