Flashbake Automates Version Control For (Nerdy) Writers
Save snapshots of frequently-updated files over time with programmer-strength version control—but without the learning curve. Free software Flashbake captures working files and adds details like the weather and your Twitter status to each version automatically.
What’s Version Control?
Subversion is an open source version control system that keeps snapshots of changing files over time. Git is the new kid on the version control system block. Git’s gotten quite a bit of attention among software developers partly because its creator, Linus Torvalds, also brought you the Linux operating system.
Primarily, software developers use version control systems like Git and Subversion to archive, manage, roll back, and merge code over time. But you can use version control for any kind of file that’s plain text—whether it’s code, creative writing, a to-do list or notes. That’s exactly what science fiction author and blogger Cory Doctorow uses it for, but he didn’t want to manually commit files by hand, like coders do. In response to conversations with Doctorow, programmer Thomas Gideon cooked up Flashbake, a simplified interface to Git that runs in the background and commits versions quietly as Doctorow writes, no intervention required. (Here’s the whole story of Flashbake in Cory’s words.)
As a programmer-turned-writer, Flashbake tickles me pink, because it makes a powerful developer tool available to anyone who’s constantly typing into plain text files. Once you start managing code with Git, you realise pretty quickly that it’s the cat’s pyjamas. But there’s a learning curve and lots of commands to master before you can really get good use out of it. Flashbake gives writers all the benefits of Git in a single command.
What Flashbake Does
In a nutshell, Flashbake does two things quickly and easily:
- Flashbake commits text files you specify to a local Git repository. You can set it to run quietly in the background, while you work, so it can do this without intervention.
- With the right plug-ins enabled, Flashbake automatically embeds ambient information in each version of those files—like the weather, your Twitter status, the last track you listened to—by including it all in each version’s commit message.
What you get is a database of file versions over time, with notes attached that tell you what you were doing, what it was like outside, and what you were listening to at the time. I used Flashbake as I wrote this very article, and pushed my local Git repository to GitHub, a free Git host, to show off Flashbake in action.
Here’s what it looks like:
Check out a history of changes to this post over time for yourself: flashbake_automates_version_control.txt / Commit History. Click on the “Commit” link to see a color-coded file diff between that version and the earlier one, like this:
I configured Flashbake plug-ins to include San Diego’s current weather, my computer’s timezone, my Twitter status, and my recently-played tracks from Last.FM (using the feed). Here’s what one such automatic commit message looked like from an early version of this post late last night:
You know you want to try it yourself. But first, a warning.
Nerd Alert: Who Flashbake Is Really For
Flashbake is a command-line system for advanced users, and requires a Linux-like shell like Cygwin for Windows or Mac OS X’s built-in Terminal. It is most definitely not for folks looking for something like Microsoft Word’s versioning. It is, however, for people who make heavy use of plain text files, don’t mind firing up the terminal and running a script or two, and know what cron is. Since Flashbake is an interface to Git written in Python, you’ll need all three installed to get this party started. Not scared off yet? Cool. Let’s go.
Install Flashbake
Flashbake’s installation and configuration documentation is already pretty complete, so I won’t duplicate it here. In short, to run Flashbake, you’ll need superuser access and the following installed as usual:
Once you’ve laid that groundwork, download the latest version of Flashbake and extract it. In the extracted directory, run:
sudo python setup.py install
Now you’re ready to start your project. Create a new folder—I called mine write—and cd into it. Then run:
git init
Make the first text file you’re going to track with Flashbake, like my_novel.txt, and type a few lines. Now you’re ready to start tracking its revisions with Flashbake. Create a new file in your project directory called .flashbake (don’t forget the starting dot) and add the name of your text file to it. For example, my Flashbake project tracks two files, called my_novel.txt and this post in a file called flashbake_automates_version_control.txt. So the bottom of my .flashbake configuration file looks like this:
# “Hot files” for flashbake to commit, add yours here my_novel.txt flashbake_automates_version_control.txt
(Flashbake ignores the first line, which starts with the #, because it’s a comment.)
Now we’re almost ready to run Flashbake. First we’ve got to configure a plug-in or two.
Shake and Flashbake Plug-ins
Flashbake plug-ins are the secret sauce that make it a worthy interface to Git. Plug-ins automatically add information to each version’s commit notes, like feed items, the weather, and timezone.
Here’s a full list of the default plug-ins that come with Flashbake. I used four plug-ins in my Flashbake project: weather, timezone, Twitter, and the feed plug-in. As the Flashbake plug-in documentation explains, list the plug-ins you want to use in your .flashbake file and its additional parameters below.
For example, my full .flashbake file looks like this:
# List what plug-ins Flashbake should invoke plugins:flashbake.plugins.weather:Weather,flashbake.plugins.timezone:TimeZone,flashbake.plugins.microblog:Twitter,flashbake.plugins.feed:Feed # Weather plug-in configuration weather_city:San Diego # Feed plug-in configuration feed_url:http://ws.audioscrobbler.com/1.0/user/ginatrapani/recenttracks.rss # limit is optional, defaults to 5 if unused feed_limit:3 # Twitter plug-in configuration twitter_user:ginatrapani twitter_limit:3 # “Hot files” for flashbake to commit, add yours here my_novel.txt flashbake_automates_version_control.txt
Note that the order in which you list the plug-ins matters. I switched weather and timezone’s position to see the weather at the top of each commit message in GitHub’s version list, here.
Now you’re ready to run Flashbake. To try a “dry run” of Flashbake and see what it will commit without actually performing the commit, cd .. to your project directory’s parent and run the following (Where “your_project_directory” is the name of your project directory.):
flashbake -d your_project_directory
You can even add a -v switch in your command to get extra-verbose output for debugging.
You’ll get output similar to this. Tweak your .flashbake file until the output is to your liking.
Once you’ve got Flashbake’s plug-ins working the way you want, simply run the following to save the current state of the file as a version:
flashbake your_project_directory
To make Flashbake run automatically in the background, use your system’s cron scheduler. Here’s the full list of ways to run Flashbake.
Publish Your Flashbake Git Repository to GitHub
Now you’ve got a local Git repository of file versions. Great, right? Well, not if you can use ‘em easily. Git newbs don’t know all the commands to search and diff between versions, but luckily a pretty web hosting solution for Git means you don’t have to know the commands. I published my local Git repository to GitHub to show off my file changes in this post. Register for a free account at GitHub and create a new repository there. Then, push your local repository to GitHub for cloud backup and a good-looking web interface to your project. Use GitHub’s setup instructions to do just that. After I created a repository on GitHub called write, GitHub told me to run the following commands:
Where “existing_git_repo” is your project directory. The git push origin master command is the magical command that pushes your local files to GitHub. By default, you have to do that remote push by hand, but if you’re automating Flashbake’s runs, you can make local commits automatically push to a remote host (like GitHub) using this handy command.
Flashbake is still a very young mechanism for driving Git, and more plug-ins and configuration options are becoming available, so we’ve only scratched the surface here. But adopting programmer version control for writing, lists or notes is something more and more tech-savvy folks will do going forward. Thanks to Leo Laporte and Cory Doctorow for making me aware of Flashbake in this episode of This Week in Tech.
Have you given Flashbake, Git, or GitHub a try? Post links to your projects and let us know how you like it in the comments.
Gina Trapani, Lifehacker’s founding editor, loves using Git for writing as well as coding. Her feature Smarterware appears every Wednesday on Lifehacker.
- Next Post: Increase Productivity By Training People To Use Your Inboxes »
- « Previous Post: Gmail Compactor Shrinks Left-Hand Links Into Icons
Comments (AU Comments | US Comments)
@brodiemac: You would probably have to treat the files as binary (as opposed to text). Any decent version control package can do it, Git included. The drawback is that you don't get neat little diff files showing what's been changed between versions, usually just a simple "file has been changed."
Haven't used Flashbake, and I didn't see a reference to binary files during my quick visit to the website, so I dunno whether or not it is built to handle binaries.
taodude
@brodiemac: I'd second the call for an automatic version control of word and excel files.
ShouldBeStudyin
@Charles: i used filehamster many years ago and then when I wanted to migrate away from it, I remember I had to purchase some plugin of theirs to do mass exporting. otherwise, all of my data was stuck in an impossible to use directory structure with versioning files all around.
mahumphrey
it's obvious Gina put a lot into this, but wouldn't it be great if a program could save any file (not just plain text) as soon as you save changes to it, and make multiple versions of that file available from anywhere. enter syncplicity, live mesh, dropbox....
i realize the comparison isn't 100% but any of these solutions would be more applicable and infinitely easier to setup.
mahumphrey
I've used FileHamster while I was completing my Master's degree, and it was very helpful for tracking all my documents I was writing for classes (.DOC, .DOCX, .TXT, .XLS, .XLSX, etc). It's a free program with a neat interface.
As a programmer I liked the idea of revision control for my documents (in case I really screwed up part of my document), but as a very busy student I appreciated the slick and easy interface that I didn't have to study to be able to use. And it's a free app!
FileHamster is available from [www.mogware.com]
Charles
Excellent article Gina, thank you. But what if I am looking for Word or Excel versioning? Can you recommend something for that?
brodiemac
I've been using an svn server to manage a repository of all my writing for about three years now. I can't even begin to describe how useful it has been for all my college papers and various writing exercises! I can back up all my work easily, manage revisions, and save myself in dire circumstances (like a crashed hard drive). Consequentially, using an svn server, i can checkout a full copy of everything I've done easily to another computer. In this way, I can 'pick up things where I left off' on my laptop when I need to do some work at school without having to deal with an arduous file copy process; just a simple "$ svn update"!
rdw200169
@jungledisk, which basically enables you to mount an Amazon's S3 bucket as a local disk drive. It will keep revisions of your files based on criteria that you set. It isn't free, but it's awfully cheap, and tremendously useful to me.
@Daegalus: looks like all the Git interaction is only one file , so should be simple to augment with support for other version control apps
@taodude: Have you ever tried the built in Version control for Word? It's available under the File menu (Office 2003).
@Sam Jones: Git is a good choice for writers because
1) its offline
2) it can go online when needed
dropbox et al are awesome , but what if you aren't connected to the internet , you loose versioning ? no thanks
@mahumphrey: you could do that auto push at file save , let me look at the flashbake source :P
Dropbox ([www.getdropbox.com] to give me more storage that I'm not using) will give you undo functionality. Google Docs also does versioning, and if you install Google Gearss, you can run without a web connection.
Git does things like document merges (doesn't work for non-text formats) and maintaining branches. For software developers, that's really important. For writers, unless you had many (more than five, I'd say) people working on overlapping documents stored in a text-friendly format, Git would not be a good choice.
Sam Jones
Hmm, I wonder if I should take this, and alter it to work with Mercurial, since I much prefer Mercurial over Git. I'll think about it, but otherwise, great idea and I might make use of that if I can get it to work ontop of Mercurial (which is also Python-based)
Daegalus
@coderneedsfood: Heh. I had the same thought. You may want to de-couple remote push from local commits, though. It might make more sense to add a crontab entry with a more coarse grained frequency to automate pushes. Maybe make it configurable via flashbake's mechanisms for parsing control options out of .flashbake files. I welcome suggests, of course, or better yet patches.
Thomas Gideon
@coderneedsfood: See my main comment below. You are exactly right, the git handling code is pretty well centralized. The plan is to eventually re-factor it into a plugin and provide a protocol for plugins to drive other source control systems.
Thomas Gideon
@taodude: flashbake doesn't have any special handling of binaries. You hit the nail on the head, the question really is more what the underlying source control system can do. As I note in my main comment below, the emphasis in flashbake is in the contextual information it assembles for the commit, less so on how it automates git.
In the head of the sources, there is a new kind of plugin for handling hot files of known types. The first plugin of this kind will become available in the forthcoming 0.25 release. The plugin will add handling specifically for Scrivener project bundles.
If someone can think of something meaningful to do in the way of pre-processing or diff'ing MS binary formats, then it should be reasonable to write a file plugin.
Thomas Gideon
Gina, thanks for the wonderful write up. It is worth emphasizing that the focus of flashbake is on capturing that ambient information you mention which is increasingly more available and rich as more people start using more social services. My thought process with the plugins was to make pulling in new bits of information simpler to help support that primary focus of the software.
Cory does such a good job of capturing the idea when he refers to his discussion with an archivist. It is about versioning, sure, but also about the "digital marginalia". I have already planned to introduce source control plugins in a future version to address some of the feedback about the initial choice of git. As one commenter notes, all of the git code is centralized. This was intentional.
As it is written, today, the fact that it can be simply invoked from the command line to just output context info, the -c switch, is also intentional. This makes it easy enough to wrap up in any number of other scripts that fit with different ways of going about writing.
Of course, the fact that the current version is released under a GPL license was also motivated by my desire to let people hack it into whatever form worked for them. One idea Cory had was a gedit plugin which may still happen. No doubt the uses to which the core context functions of flashbake could be put are only limited by the number of tools and processes out there.
I have some less well formed thoughts about how to make the tool even more friendly for less technically inclined users. An interactive configuration tool has been suggested as has a simpler graphical history viewer more tailored to writers and other creatives who aren't as interested in some of the info hackers will want out of a source control tool.
Stay tuned, the tool is still under very active development. Of course, since I host the project on github, anyone is welcome to fork and do what they wish under the terms of the GPL license. I've already accepted some changes back into my fork as indicated on the credits page of the wiki.
Thomas Gideon
@scokar: Word 2007 took away the built-in version control. Now all you can do is use the "track changes" feature, which isn't quite as simple.
It's probably my only real complaint about Office 2007.
@D Wiz: It's just nerdy goodness. I can see why some writers would be interested in this sort of feature, though you're right, there's no huge need for the extra info. (Cory Doctorow is definitely a writer/geek who'd be jazzed about this tool.)
Most writers aren't programmers, so tend to think a little differently... ;-)
Mercurial Chimaera
Anyone know if something like Flashbake is available as a nerdy front-end to Subversion (SVN)?
Mercurial Chimaera
@brodiemac: As already mentioned, you'll have to store the files in binary mode; most version control tools have a special binary mode or deal with binary files automatically.
One trick I've used to get around the diff issue with binary files is to store my Word files in rtf format. Since my revision control needs were modest -- no collaborators at home -- RCS was adequate (and RCS doesn't do binary, not easily, at least). But I'm very intrigued by Flashbake and it's nerdy goodness... think it's finally to migrate to a more robust and complete VC system.
Mercurial Chimaera
@earth2marsh: i also use jungle disk and find its delta backups and versioning not only key but cost effective as well. the JD client however does not do backups on file saves, but only on a timed schedule so i didn't mention it.
mahumphrey
Maybe I am missing something... Why is having the weather, twitter updates and last song listened to embedded in a revision useful? To me it seems like a "cool" feature that doesn't really provide any value. What's so hot about it?
D Wiz
God, I hate that I'm "that guy," but I have to point out that Linus Torvalds brought us not "the Linux operating system," but the Linux kernel that GNU/Linux operating systems use. While references to the entire operating system as just Linux or the work of Mr. Torvalds are relatively common, they are offensive to some of the people who put in a lot of work before the kernel was even a glimmer in Linus' eye.
See what RMS has to say about it: [www.gnu.org]
This issue can sound nit-picky, but I wouldn't bring it up here if I didn't think it was An Important Issue.
More on-topic, Flashbake looks like an excellent program and I'm looking forward to trying it. I'm also excited to send this tutorial to my father, whose a writer... it's a great tool for him, and now it's been explained in terms he can totally wrap his head around!
parkerhiggins
@Matthew Bergman: maybe thats why the twitter / audio / weather is recorded ? , hmm what was i listening too when i wrote that part :P
I'm sorry I know he put a lot of work into this system but this whole thing is an abomination for what version controll is supposed to do. You version a file. You do discrete commits not a commit every 5 minutes. It's impossible to find anything that way. Imagine a novel with 5000 commits. Also this actually seems much more complicated then simply installing git.
Matthew Bergman
@D Wiz: I think I heard somewhere that having those information saved could help the writer determine/guess sometime in the future what his/her mood was when particular changes where made, or something.
yriafelc
@Mercurial Chimaera: hack the source !
i keep getting an error when i run:flashbake -d mydirectory "Error: No plugins configured" my .flashbake file only contains :test.txt
AlariceBabalas
Can't stop reading "Fleshbake". :/
WadePaullus
@brodiemac: It isn't automatic and it isn't "in the cloud" unless you place it there. My company has been using Alfresco for document management. For a single person, it is overkill.
You can use the normal software version control systems for binary files, they just aren't very efficient.
Don't know that I'd recommend placing **everything** into the GitHub. Some things are meant to be private.
TheFu
@yriafelc: That makes some sense. Still, I think Gina makes a waaaay bigger deal out of it then it is worth. I think the version control is awesome but those other features seem like a waste of time. I guess they weren't created for me. :-)
D Wiz
I used to use Aladdin Flashback for this sort of thing. It automatically grabs your file whenever you save it, in any application, and creates a compressed version. You specify how many versions you want to keep for a specific file. It is really great if you are the type that forgets and does a Save instead of a Save As after major updates but really needing to retain both versions. I think this is compatible with XP, but haven't tried it on Vista. It does take up space on the taskbar; it doesn't minimize to the system tray or run hidden.
VihoHecteyes
@brodiemac:
I know there is pretty awesome integration for this with SVN in word 2k7. I only used it once so I'm not totally familiar with it as I usually just use SVN for code and accidentally added a specification document to the repo one time which is how I found about it.
I was using SVN w/ the tortoisesvn client btw.
Bryan Petzinger
You may want to try History Explorer. I have been using it via giveawayof theday.com for illustrator and indesign files and it works flawlessly and more importantly brainlessly. http://www.exendo.se/history-explorer-simple-version-control....html
ZanebonoCroesus
Well perhaps I've been using version control for too long, but I don't see what's so hard about `add`, `commit`, `update`.
Anyway, why the hell would you want the weather saved with your revisions?
I have git set up on a cron job like this for my PhD work. Sure it doesn't have the advantage of discrete named commits, but (1) it does keep your changes and (2) it does so without you having to think about it and (3) if you combine it with meaningful commits once in a while, you can still diff those with each other, and all of the automated ones in between don't really hurt you.
SakhiLabadoozie
@brodiemac:
Word and Excel have "Track Changes" built-in, don't they?
@mahumphrey:
You shouldn't have to save changes to the file at all.
Any time you make an edit, it should save the changes continuously and automatically in the filesystem. You can consciously press "Commit" to save a certain revision for easy finding later, like bookmarks, but it should save automatically, too.
I wrote what I imagine here:
[www.endolith.com]
Why wouldn't branches be important to writers? Think of it as "undo/redo" trees instead of archived versions of a linear progression.
Google Docs revision display sucks, and anything in "The Cloud" is going to be inherently slow.
Yes, the operating system is properly called "Ubuntu", not "Linux" or "GNU" or anything else. ;)
@D Wiz: I don't understand the point of this, either.
Versioning should just be built directly into the filesystem.
The cost of storing diffs for each file is tiny compared to the size of a modern hard drive, and you'd never lose your work ever again (not even work you overwrote and then changed your mind about).
[www.endolith.com]
I'm a writer, but for years now I've managed writing projects in git repositories (and subversion before that). I quite adore it, and think that this is the right interface for a lot of writers who might not think about "commitable milestones" in quite the same way that programmers do, and in a lot of ways I think that having metadata is of archival interest so that's pretty nifty.
Having said that, I should look at getting a pre-commit hook that adds in weather/timezone, twitter information, and so forth, to the commits that I already do to provide context on a non-automated framework.
For OS X users, consider using launchd rather than cron. It works a bit better.
Crikey! You're right about it being nerdy. I zoned out by the third par.
Just give me WordPerfect 5.1 running on Vista and I'll be sorted.
I've used Filehamster too for several months. Great idea but i had to stop using it due to it's inability to handle really long path names and large files collections.
OrtensiaDavidson