Gmail Product Manager on IMAP and Greasemonkey
Posted by Gina Trapani at 4:00 AM on November 20, 2007

Just when we thought that Gmail was stagnating, the big G rolled out upgrades that delighted power users: IMAP access, and a speedier, developer-friendly interface. As makers of the Better Gmail Firefox extension, we were psyched to get a personal heads-up from Gmail's Product Manager, Keith Coleman, alerting us to the revamp so we could update the extension. In addition to wrangling third-party developers to update their code, Coleman took time out of his busy day to answer some of our burning questions about the recent changes at Gmail. After the jump, hear what he has to say about IMAP, LDAP, Greasemonkey, the iPhone, and other third party Gmail clients, apps, and development.
Lifehacker: Gmail's IMAP support is a HUGE move forward for web-based email, something no other free webmail has offered. What made Google decide to offer IMAP? Is there any possibility of LDAP access to contact information in the future? (Editor: LDAP works similarly to IMAP in that it syncs your contacts to your address book, versus just exporting them.)
Keith Coleman, Gmail Product Manager: One of our core philosophies at Google is that users' data should never be held hostage. In the case of Gmail, users should be able to access their mail however and wherever they want. We started down this path soon after Gmail launched—in 2004 we added free POP access and auto-forwarding. POP let users get their mail on mobile devices and desktop clients. Auto-forwarding let people do useful things like filter mail and forward the most important messages to their phones, and even made it easy for users to switch away from Gmail to another email service if they chose. Millions of people start using Gmail each month, and many of them tell us how hard it can be to switch email accounts, particularly when switching from a service that doesn't offer POP, IMAP or forwarding. We didn't want Gmail users to have to go through that if they ever wanted to move away.
POP and forwarding were a nice start, but the downside of POP, as you know, is that you often end up having to read your mail twice—once on your mobile device or desktop client and again on the server. IMAP solves this by keeping mail in sync across all clients and devices. It provides a good user experience in classic email clients, so for webmail services, particularly those that rely on ad revenue, there's a risk that users may switch away from the web interface to desktop clients. Our belief is that if we make the Gmail web experience good enough and fast enough, people will choose to use it over other clients. This was an unknown when we launched POP, but it turned out to be right—most Gmail POP users use POP in addition to, not instead of, the web interface. We've taken the same approach with IMAP, giving users more choice and better email experience.
Your LDAP suggestion is an interesting one. We currently offer a way to import and export contacts, but don't yet provide an API like LDAP.
Lifehacker: That's interesting—counterintuitive, even—that users visit Gmail's web interface in addition to (rather than instead of) getting their email via POP. Thing is, POP's been offered by lots of web-based email services for years now, but not IMAP, and IMAP is not new. From what I understand of email-fetching protocols, the nature of IMAP is that it's more resource-intensive than POP (due to the constant reading and writing back to the server). So I always assumed that was the reason why no free web-based email service offered it. Do you think that's true? Or was it a matter of timing—did the iPhone launch play in?
Coleman: IMAP is something we've wanted to launch for a long time. It is a more complex protocol than POP, so took some time to implement, particularly given some of the work we did to make IMAP folders map well to Gmail labels. Resource usage wasn't a major factor in the decision to launch it—we knew a lot of users would love IMAP (including many new iPhone owners) and were eager to get it out as soon as it was ready.
Lifehacker: Gmail's recent upgrade not only improves performance, but it includes a Greasemonkey API, virtually inviting developers to modify Gmail's interface. As a Gmail extension developer myself, I'm thrilled! But I imagine Greasemonkey scripts and extensions only cause more headaches and work for the Gmail developers, and only benefit a small, demanding, power user base. Why did Google decide to go this way?
Coleman: We like to see and support innovation. Greasemonkey is a powerful tool in that it lets developers extend any web site. The downside, of course, is that because it extends sites by directly modifying the source code, rather than using stable APIs, extensions written in it are fragile to changes in the source code, and can even cause the host sites to break. As you point out, this can definitely be a headache, particularly when users hit bugs and can't tell whether the site is broken or the extension is broken.
It's inevitable that developers will want to use Greasemonkey to extend the products they use and like, so we thought we'd try to help make their extensions a bit more robust by providing a more stable API. The Gmail Greasemonkey API is still experimental—we're going to see how it's used and how helpful it is.
Lifehacker: Does the experimental API help Google track how it's being used?
Coleman: Right now it is purely a set of convenient wrappers on top of the code.
Lifehacker: You contacted developers (including myself) before the new Gmail upgrades rolled out to help us prepare our scripts and extensions for it. About how many developers were in the loop? How did you choose who to contact?
Coleman: Google is a data-driven company, so normally in situations like this we'd measure the number of active users running each extension and focus on those most frequently used. With Greasemonkey and other Firefox extensions, we don't have the ability to measure usage, so we did our best to identify the most popular extensions. We scanned through addons.mozilla.org, userscripts.org, and blogs that mentioned Gmail extensions. It wasn't always easy to find contact information for the developers, but we were able to reach some of the authors and give them early access to the new code.
Lifehacker: My fellow developers may kick me for asking this, but I must. Extensions and user scripts that do things I imagine Google doesn't love—like hide ads, or use Gmail's storage as a file drive—have never gotten a takedown notice from Google, as far as I know. Any comment on that?
Coleman: We like to see innovation—users trying new things—though of course don't condone violations of the Gmail terms of use. We're particularly concerned with abuse of the service that could affect the experience of other users.
Editor: For the record, Better Gmail does not include scripts that violate Gmail's terms of use.
Thanks so much to Keith for taking the time to give us a little peek behind the curtain at Gmail.
Tags: email | exclusive lifehacker interview | feature | gmail | greasemonkey | imap | iphone | top

Comments (AU Comments · US Comments)
There are currently no AU comments for this post.
jflaming
Posted 2:44 PM 19/11/07
@leo
#4 has been driving me crazy forever. Thanks for bringing it up. I constantly miss emails when searching because of this.
jflaming
leo.babauta
Posted 2:08 PM 19/11/07
Great interview, Gina! A couple things I'd like to ask Keith:
1. What should we expect in the next batch of new features for Gmail, and when should we expect them?
2. When are they planning to have Gmail work with Google Gears?
3. When (if ever) will they allow you to sort by sender, date, and especially size of email?
4. Why does Gmail's search not work like Google search when you enter a partial term ... like when you search for "trapan" it won't turn up emails with "trapani", for example.
leo.babauta
earth2marsh
Posted 2:02 PM 19/11/07
Here's the Greasemonkey API docs, btw.
earth2marsh
earth2marsh
Posted 1:59 PM 19/11/07
Hey, it might be nice to have a link to the docs for the Greasemonkey API so we don't have to Google it! ;)
earth2marsh
Roriniho
Posted 1:47 PM 19/11/07
@leadster618: No, I use it sometimes, but not at the moment. It isn't on there either.
Roriniho
tut
Posted 1:30 PM 19/11/07
um...actually AOL mail has had IMAP access for a while, so gushing about "something no other free webmail has offered" is a bit off base.
tut
nycGRAEME
Posted 1:23 PM 19/11/07
hello Google hottie! :)
nycGRAEME
leadster618
Posted 12:48 PM 19/11/07
@Roriniho: Are you on Opera? I just realized I have the new version on firefox, but I am on Opera and its not supported apparently... so there's a question for google...when will you update for opera? I use Opera because of less resources and I think google should realize that people don't just use ie and firefox... oh well i guess i'll keep waiting...
leadster618
Roriniho
Posted 12:44 PM 19/11/07
@leadster618: I agree: I still haven't gotten IMAP access, and neither have I got thre new interface. Come on google, give the new stuff to everyone, not just a select few. >..<
Roriniho
leadster618
Posted 12:34 PM 19/11/07
Ah but the real question is: when will Gmail start giving 2.0 to all its users... I've yet to see the famed "newer version" in the corner... and there isn't an "older version" up either so I don't already have it...
After reading these great articles on gmail, i moved my mail from yahoo and another account to gmail, got the extension for firefox and now I'm just waiting to get the go ahead...
so is there a date for everyone? or does it depend on when you started gmail? since my gmail is only a few months old, will I be waiting a while?
leadster618
GlennA
Posted 3:49 PM 19/11/07
About using both the web interface and a POP client... I've been using Thunderbird for composition for some time now since the web interface is somewhat limited in options/features for composing; not always, of course, but quite often. So, yeah, using both makes perfect sense to me.
GlennA
sinderphytik
Posted 3:37 PM 19/11/07
@leadster618:
@Roriniho:
I'm pretty sure they've said that they've gone ahead and enabled the new version for everyone...you just have to be using IE7 or Firefox 2. (works for me on IE7, not on IE6...well, Maxthon, but IE rendering engine)
sinderphytik
Cidinho
Posted 3:02 PM 19/11/07
@jflaming: No good with exampl*? I guess it doesn't work like Google because I believe a lot of people just have similar content on all their mail.
Cidinho
Kuz
Posted 3:00 PM 19/11/07
5. Why does GMail's search not work within the contacts? i.e. you cannot search your contacts for terms besides name and userid, like "New York" or "February" or "basketball."
Kuz
Andrew Heiss
Posted 5:02 PM 19/11/07
Please get that LDAP support working soon! That would perfect Gmail in every way!
Andrew Heiss
nicopolitan
Posted 6:57 PM 19/11/07
So... when did they say it was coming out of beta? ;)
nicopolitan
Andrew
Posted 6:25 PM 19/11/07
I'd really love a way to make it so that if I'm using IMAP through Outlook 2007, I'm not stuck with a "Junk E-mail" label in Gmail. I suppose that's a problem that's on the Outlook end of things, but it'd still be great to disable this.
Andrew
David Hunter
Posted 2:25 AM 20/11/07
Hey has anyone else been having a problem with attachments? Since switching to IMAP I've found I can open them when they immediately come down, after that through they seem to become unreadable?
Thought I would see if this is a common experience.
Cheers
David
David Hunter
prupert
Posted 4:47 AM 20/11/07
@nicopolitan: lol
prupert
prupert
Posted 4:46 AM 20/11/07
@sinderphytik:
Sorry, the new version of Gmail ONLY seems to work if you choose English (US) as your default language, even if you are already using a supported browser.
I'm a UK user and have been waiting over half a month for the new feature, then I read somewhere that if you change your default language to English (US) you get the new features....I did it and it worked.
Discrimination, Google????? Or just slow at language translation - though why it is taking them so long to translate English (US) to English (UK) (which after all is the mother tongue, English US is just for people too lazy to spell correctly ;) ) is an unknown. The Search update for Reader came through on the same day it was announced on my account, so it is something specific to Gmail.
So, if you want IMAP and haven't got it, check your language settings...
prupert
daniel.j.doughty
Posted 9:58 AM 20/11/07
anyone else seeing problems IMAPing mail to a Windows Mobile device? I keep getting empty messages.
daniel.j.doughty
ahoier
Posted 10:34 AM 20/11/07
It's still in beta, but yea, could be a translation delay I suppose...
And sure, AOL has had IMAP for years, but who uses AOL mail?...really :P
ahoier
lmjabreu
Posted 10:28 AM 20/11/07
@daniel.j.doughty: yeah, that's a WinMo problem. HTML emails are only supported if you use an exchange server, at least according to the documentation of WM6. Sometimes I can see a partial render, but I'm guessing MS's counting on exchange to reformat html mails or something, who knows.
Hey Google, give us a gdata contacts api please, I've successfully dumped Outlook but I still need to manually update contacts on my WinMo device whenever I do so on GMail =| (though I made the initial import using outlook)
I heard there's some kind of xmpp access to the contact list tho.
lmjabreu
Tom Ulrich
Posted 12:46 PM 20/11/07
@daniel.j.doughty: Oh yeah, there are a bunch of threads on Google Groups (see here [groups.google.com]).
Tom Ulrich
nlurker
Posted 5:57 PM 20/11/07
I am using Firefox 2, I have my language set to English(US), and I don't have the new version yet.
nlurker
cphillips51
Posted 4:40 PM 23/11/07
This has been a revelation on my iPhone. Switching from POP to IMAP, and getting access to all the folders I regularly use on gMail, really helps speed things up when I'm on the go. Plus, I don't have to keep on constantly marking stuff I've already read on my computer as read on my iPhone anymore, which is also a huge boon.
cphillips51