Migrating Bare Bones Yojimbo to WordPress

In a previous post, I wrote about exporting data from Bare Bones Software’s Yojimbo and using Tomboy as an alternative. My migration script scraped the content from Yojimbo Sidekick and wrote XML files in Tomboy Note format. Though there were some drawbacks, such as tags being unavailable in Yojimbo Sidekick, I thought Tomboy’s search feature would be adequate. A couple of weeks trialling Tomboy proved that it wasn’t going to be a Yojimbo killer. Tomboy can’t compete in terms of overall usability and though it’ll work on my Linux and OS X machines, note synchronisation takes some setup that didn’t warrant further time investment.

Once again I turned to WordPress as an easy solution. There’s a risk of seeing WordPress as my hammer for everything that looks like a nail but it has taxonomies, a reliable-enough search functionality and being web-based, works across all my devices. I’m very familiar with the platform and have already built up my own set of tools to export and migrate content. Why not use WordPress? Getting data out of Yojimbo was another issue. The quick and easy Yojimbo Sidekick route already proved inadequate so it was time to dig in and reverse engineer Yojimbo’s storage mechanism.

Analysing and exporting the Yojimbo database

‘Reverse engineering’ turned out to be too lofty a term for the task. It was obvious after quick look that Yojimbo uses an SQLite database to store information. Firing up DB4S to analyse the tables and bit of analysis revealed the tables, columns and relationships that are important for exporting our notes. The columns are a little oddly named but it didn’t take long to figure out the necessary fields for migrating to WordPress.

Table: ZITEM
Column Description
ZBLOB This looks like an ID
ZNAME The Yojimbo note title
Z_PK The ID to the Z_15TAGS relationship table
Table: ZTAG
Column Description
Z_PK The Tag ID
ZNAME The tag name
Table: ZBLOBLSTRINGREP
Column Description
ZBLOB ID
ZSTRING String for unencrypted item
Table: Z_15TAGS
Column Description
Z_15ITEMS1 Relationship ID
Z_25ITAGS Tag ID

Yojimbo SQLite tables

It became a simple matter of tweaking my Drupal to WordPress migration queries to extract from the Yojimbo database and create WordPress posts. Unlike with the Tomboy Notes route, it was possible to recreate the tags, which is what makes Yojimbo so useful. One drawback is that I haven’t figured out how to extract encrypted notes but I didn’t use Yojimbo to store important encrypted information so that wasn’t a priority.

WordPress as a Yojimbo alternative

Using WordPress as a Yojimbo alternative might not work for everyone but after several months use, I’ve found it to be an excellent cross-platform replacement. The installation and database runs on a NAS drive connected to my local network so is accessible to all my devices. Standard WordPress taxonomies, search and plugins makes content management simple once you’ve imported the Yojimbo content. In fact, by migrating away from Yojimo, I’ve ended up creating my own full-blown personal knowledge management system.

If you need to export your Yojimbo notes to a cross-platform alternative, give WordPress a try. You can grab my migration script from GitLab but please keep in mind that it was a quick hack to achieve a specific one-time objective. You may need to hack it to suit your own setup.

Migrating Delicious Bookmarks to WordPress

Delicious is one of those services that few people know about but loved (and probably hated in equal measure) by those who use it. Their offering is easy to grasp: Delicious is a social bookmarking service that lets you “Save, organize, and remember the links you find interesting or useful around the web.” I’m sure you’ve been in a situation where you’ve tried to remember the source for a particular factoid or needed to cite an article but lost track of the link. The obvious place to save links is in the browser bookmarks but this quickly becomes unwieldy if, for example, you’re doing research on lots of different topics. Unless you install bookmark syncing software, browser-based bookmarks are also cumbersome if you run different browsers and computers. I used Delicious to store thousands of links over the years, all of which where tagged and easy to call up when needed. They say in their about page, “Delicious remembers so you don’t have to. It’s easy to build up a collection of links, essentially creating your own personal search engine.” It was great at that and absolutely invaluable if, like me, you have a poor memory. Programmers, bloggers, researchers, anyone who needs to save a link for future recall needs something like Delicious. Founded in 2003, it’s also a veteran in an industry where anything that lasts more than a couple of years is considered successful.

Old book with bookmark for Bookmark to WordPress migration

Sadly, this glowing opening isn’t to recommend the service. You’d think its longevity would be a sign of a rock-solid, mature platform but no: Delicious’ loyal user-base has been rewarded with outages, lost data and broken features. That people persisted despite years of problems shows just how useful a service it provides. When the site went down again earlier this year (2016), I decided it was time to make a move. Unfortunately I couldn’t find an alternative trustworthy enough for several years-worth of carefully curated bookmarks. Free web services are notorious for disappearing into a black hole with all your data. As the saying goes, you get what you pay for.

After a long search, I began to realise there wasn’t anyone out there who could be trusted. This isn’t just Delicious and bookmark curation problem though. These past few years has shown that The Cloud hasn’t lived up to its promise. Data breaches, surveillance, data collection, outages, and catastrophic meltdowns; we can’t really trust a third-party service, especially if they don’t have a clear business model. I grudgingly conceded that cobbling together my own solution was the only option.

The rest of this post describes my solution, why I chose WordPress and how you can convert your WordPress site to host a personal bookmarking service. This will be valuable even for those who’ve never used Delicious but would like a way to curate links.

Exporting the bookmarks

Before doing anything else, the bookmarks need to be in form that’s easy to manipulate. Delicious had a feature to export all your links, tags and notes into a HTML file Netscape bookmark format. At the time of writing this has been ‘temporarily disabled due to server load’ (really, Delicious?). Fortunately I was in the habit of making frequent backups due to Delicious’ unreliability so my export file was only a few weeks old. I threw together a quick Python script called DeliciousPy to scrape the file and save the results into a local MySQL database. Feel free to grab the source over on GitLab.

It was a one-use tool so a little rough and undocumented. (Sorry, it wasn’t written with the intention of public release!) Anyone with basic Python knowledge should be able to easily figure out how it works. Essentially, I parse the Delicious export file using Thibauld Nion’s Netscape bookmark parser that is part of his Water On Mars! project. Everything is then inserted into three MySQL database tables:

  • bookmarks: the table storing bookmark title, url, note, creation time and privacy flag.
  • tags: all the tags associated with the Delicious bookmarks.
  • tag_relationships: the relationships between the bookmarks and tags.

After doing a pip install to install the requirements, you run DeliciousPy with the command:
$ ./deliciouspy.py -p /path/to/delicious/export_file.html

Running the script exports:

  • Bookmark title
  • URL
  • Tags
  • Note
  • Created date

Getting the Delicious bookmarks into a database opens up options for importing the data into a different platform. You can roll your own custom tool or use a content management system. I chose WordPress.

WordPress as a personal social bookmarking service

My wish list for a Delicious replacement was short:

  • Save links with accompanying notes.
  • Tagging and search to easily call up previously saved links.
  • Simple, bare-bones functionality free of unnecessary features.
  • Set links to private for personal research or public for sharing.
  • Avoid relying on a third-party service.

It didn’t take me long to settle on WordPress. I already have expertise with importing data into WordPress from my Drupal to WordPress migration service. It’s grown into a mature content management platform with a huge user and developer base, apparently powering 25% of all websites. It has excellent documentation, can run on a standard hosting package, has a simple user-interface and a flexible taxonomy system. The code is open source so if you self-host the site, you won’t run into the third-party service problems of getting locked out of your data. WordPress has all the components for a home-brew bookmarking service.

My Bookmarks plugin

I’d originally hoped to avoid building my own plugin but none of those in the WordPress Plugin Directory suited my needs. Most were too feature-packed and I wanted something lightweight: just have some way of saving and tagging links for later retrieval. Putting together simple bookmark storage isn’t difficult: ‘bookmarks’ can be nothing more than a post with a single custom field to store the URL. To keep them separate from standard blog posts, I created a custom bookmark content type and packaged it into a plugin called My Bookmarks. (I’m not very imaginative when naming things.)

Delicious has a social bookmarking aspect that lets you publish your links, follow other users and discover pages those in your network have saved. Social bookmarking wasn’t compelling for me so I didn’t attempt to include this functionality. In my opinion, publishing the links on a blog is social enough and any anyone who wants to ‘follow’ can subscribe to the RSS feed built into WordPress.

Bookmark sharing id controlled through the normal WordPress Visiblity setting. Set to Public so the bookmark gets posted on your blog; set to Private so it’s visible only after logging in to WordPress.

One great thing about Delicious was a handy bookmarklet that allowed you bookmark your browser’s current open page. Rather annoyingly, this feature broke for me at some point after it was acquired by one of its many owners. I implemented this functionality in the form of a WordPress-style Press This bookmarklet. Code from the now seemingly abandoned Linkmarklet plugin by Jonathan Christopher does most of this work.

Experience so far

The My Bookmarks plugin hasn’t been published to the WordPress Plugin Directory as of now. I’m still trialling it to make sure everything works as expected before releasing but you’re welcome to grab the source and tinker with it for your own needs. Note: As of December 2016, the My Bookmarks plugin is temporarily hosted over on GitLab.

Currently the plugin is installed on my internal WordPress-based Knowledge Management System on my local network. It serves up thousands of bookmarks from around 10 years worth of links. I replicated everything that made Delicous so useful for me without all the drawbacks of a third-party service. So far so good. The bookmarks that I’ve spent years curating are now fully under my control—and under my responsibility. Backups are still important but bookmarks are now included with my usual automated WordPress database dumps. All-in-all, I think it was a good move to finally migrate from Delicious.

Book photo by Katia Grimmer-Laversanne from freeimages.com

FeenBan: A shadowban plugin for WordPress

Michael W. Dean of the Freedom Feens Talk Radio Show was having a problem with concern trolls. He wanted a way to shadowban commenters so I made him a plugin.

Here’s what he had to say about it:

“Gets rid of trolls without them knowing they’ve been banned. They keep posting, but no one other than them can see their posts.

Deliciously devilish! Trolling the trolls!

FeenBan! I’m not just the namesake, I’m also a client!”

For more information, visit the plugin page or download it from the WordPress Plugin Directory.

Drupal to WordPress Migration Tool

The Drupal to WordPress Migration Tool exports the bulk of the content from Drupal 6 to WordPress 3.5. It runs a series of MySQL queries to read information from a Drupal database and convert the content to WordPress. Please see this post for some background on the project.

A web front end is available for very basic Drupal installations. Sample SQL queries are included for more complex setups. It is an open source project released under The MIT License and hosted on GitHub.

The Drupal to WordPress Migration Tool shows how I like to structure code but since it’s an on-going project, it should be considered alpha code. (It uses HTML, JavaScript, jQuery and PHP but should be considered alpha code.)

Now accepting Bitcoin for payments

I’d installed a Bitcoin wallet back in 2010 but at the time there wasn’t much of a market for Bitcoins. With all the publicity the technology has been getting lately, now seems like a good time to do some more experimentation. Part of that includes seeing if it’s practical to accept them as a form of payment.

The exchange rate is quite volatile right now so I’ll start off slowly and open this up to selected projects only. If you’re interested in paying for your project in Bitcoins, please drop me a note and we’ll work out the details.

For those who are interested in Bitcoin and would like to know more, here’s where to start:

(1) Get an overview at

(2) Install a BitCoin wallet. I went with Bitcoin-Qt.

(3) After installation, wait for your wallet to sync with the network. Warning: this could take days!

Drupal to WordPress utility now public

I’m releasing my in-house tool for Drupal to WordPress migration. You can go directly to the download page for the latest version of the utility.

If you’d rather let someone else do the work, you can find more information at my dedicated migration service page.

When I started Another Cup of Coffee, we specialized in setting up Drupal since it was—and still is—a very versatile content management system suitable for anything from simple ‘brochureware’ sites to online shops and social networking. One of my working assumptions was that website owners would appreciate a platform that offers more flexibility as their needs expanded. However, over the past few years, I’ve received quite a few requests from new and existing clients to migrate sites from Drupal to WordPress. At first it seemed a little odd that it was almost always in this direction and rarely the other way around.

Some investigation quickly brought out an obvious answer: there is a Drupal module called wordpress_migrate that supports migrating WordPress blog exports into Drupal, but no simple mechanism to convert Drupal nodes into WordPress posts and pages.

After many years of experience with running Drupal sites, I can understand why a certain type of website owner would prefer WordPress. It’s just much more suited to the non-technical user who just wants a simple site with fairly standard functionality. (Incidentally, this describes our target market.) Since I’d first started using WordPress in 2004, it has evolved from simple blogging software into a fully-fledged content management system. Further, it’s more user-friendly out-of-the-box than Drupal’s default utilitarian interace and has a simpler update system. From my own experience, Drupal maintenance can be quite demanding with its frequent core and module updates. WordPress, on the other hand, is refreshingly easy to maintain.

Releasing our in-house utility

Unfortunately for us, manually moving a client’s site from Drupal to WordPress was quite tricky and time consuming, especially when lots of content types and entries are involved. We obviously needed some sort of automated tool but to my surprise, all that I could find online were either articles about people experimenting with SQL scripts, or paid-for services like GoWordPress and cms2cms. Our margins are pretty tight and paying for the service just wasn’t an option so I decided to hack together some scripts to build our own in-house tool. A blog post by Scott Anderson at Underdog of Perfection provided the best steps to get me started.

After using our tool to migrate a fair number of sites, I realised that it was time to give back to the community. After all, my code built upon the efforts of others who freely shared their work. I’m therefore releasing our Drupal to WordPress Migration Tool for public use under the MIT License.

Screenshot of our Drupal to WordPress migration tool

Where to download the Drupal to WordPress Migration Tool

I’ve set up a dedicated page for downloading our Drupal to WordPress Migration Tool which will have the most up-to-date version. More feature-rich versions can be found there as feedback comes in and bugs are reported. The first version should definitely be considered a ‘beta’ and only used by people who know what they’re doing when it comes to setting up and configuring content management systems. Currently it supports only Drupal 6 and WordPress 3.5. Eventually I aim for it to be a ‘wizard’ style utility that can be used by less technical people, like web creatives who’ve been tasked with migrating their clients’ sites from various releases of Drupal into the latest version of WordPress.

If you’re unsure of how to proceed or would simply like someone else to do the work, please drop us a note and we’ll be happy to provide a quotation.

Go to the download page. IMPORTANT: I’m offering this tool with no warranty or support implied.

Another Cup of Coffee Limited

We support Drupal and WordPress websites for small businesses, media agencies and not-for-profit groups. Our work includes setting up and maintaining content management systems, developing custom code and troubleshooting problems with servers.

My Role as founder

Founder, project manager and developer, 2006 to present

Day-to-day, I manage remote teams for my company, Another Cup of Coffee, using a simplified version of Scrum. We’re a group of independent professionals from different cities around the world working under one brand-name. When a new project comes along, I put together a team with the most relevant skills.

Aside from managing projects, I’m also a hands-on developer and systems administrator.

Arcware Solutions Ltd

Arcware Solutions Limited (ARC) offers IT and business process outsourcing services. It has partnered with suppliers in South East Asia and manages the entire outsourcing process.

Testimonial from Ray Catindig, co-founder:

“Highly industrious and knowledgeable about the Technical landscape and, most importantly, maintains a high level of moral fiber.”

My Role as a co-founder

Project Manager and Technical Advisor, 2003 to 2007

My role in ARC is divided into several areas:

  • Project manager for client projects.
  • A manager for the company’s IT infrastructure.
  • A technical advisor when bidding for projects.

Most of my time is spent working with clients and clients to assess what a project involves. Together, we identify what are the business and technical requirements; select suitable suppliers; and provide specification documents.

Web Developer

I re-coded our original static website it for W3C standards compliance and accessibility using WCAG. The website was optimised, reducing average page sizes from 300k to under 120k. Technical skills important for the website project were:

  • XHTML
  • CSS
  • PHP
  • Apache
  • Linux
  • Knowledge of W3C standards
  • Knowledge of WCAG

Briliant UK Ltd

Brilliant UK Ltd specialised in e-commerce websites and Content Management Systems. It was part of the Pod1 Group, a Top 50 interactive agency.

Testimonial from Usman Sheikh, CEO:

“Anthony was an absolute pleasure to work with. He had a very clear understanding of IT infrastructures and the digital medium on the whole so his opinions and insights were invaluable to me on the various projects where we worked together. I would be more than happy to recommend his services to anyone looking for a quality IT service provider.”

My Role as a project manager

2006

During my time at Brilliant, it was a small but growing digital agency. I was the lead project manager and also managed the company’s IT infrastructure.