Migrating from Kindle to iPad: An Illustrated DRM Primer

Over the week­end I made a tweet that sev­eral peo­ple asked about:

Basically, they wanted to know how to con­vert DRM pro­tected Kindle books over to ePub books that will work on the iPad.

I love the fact that Amazon was able to release a Kindle app for the iPad on the very first day, but in a side-by-side com­par­i­son, I found Apple’s iBooks to be a lit­tle more use­ful to me. It’s a seri­ous set of trade-offs. On the one hand, the Kindle app syn­chro­nizes with the cloud and remem­bers what page I last viewed. If I pick up my actual Kindle or use the Kindle app on the iPhone, then I can resume exactly where I was. There is no equiv­a­lent in iBooks. On the other hand, the Kindle app has no dic­tio­nary, whereas the iBooks dic­tio­nary is stream­lined and non-obtrusive. Weighing the two fea­tures, the dic­tio­nary is more impor­tant to me. I can man­u­ally fig­ure out where I am in a book if I switch devices, but I often run into words I do not know — for instance “blue­stock­ing” (a deroga­tory term to describe an intel­li­gent or lit­er­ary woman) — mean­ing I have to jump out of the Kindle app, load up a dic­tio­nary app, wait for it to fin­ish load­ing, type in the word, then switch back to the Kindle app. It is just so much eas­ier to tap the word and get a def­i­n­i­tion in a pop-up bub­ble.

Given this, I started con­vert­ing my Kindle books to ePub for­mat. This is a two step process. For the folks that just want to do it and fig­ure out the details on their own, those steps are:

  • Use the “dedrm” Python script to strip the DRM from the Kindle *.azw file
  • Use Calibre to con­vert from azw (it’s really MOBI) to ePub

Strip the DRM from the Kindle file

This step is the most dif­fi­cult and takes a lit­tle Terminal mojo. The fol­low­ing direc­tions assume you are run­ning on a Mac (I am sure Python exists for Windows, but you’re on your own if you have a Microsoft box) and that you can do some basic nav­i­ga­tion around the Terminal com­mand prompt.

The Kindle AZW files are really just MOBI (a stan­dard eBook for­mat) with some spe­cial DRM applied. The DRM is a sym­met­ric cipher, mean­ing it needs a key­word to encrypt and the same key­word to decrypt. That key­word is just a reshuf­fling of your Kindle’s ser­ial num­ber.

First, you will need your Kindle, with the pur­chased book down­loaded onto it. Next, you will need a set of Python scripts that go by the name dedrm (includ­ing kindlepid.py, mobidedrm.py, and monidedrm2.py). I am not a lawyer, so I do not know if I can pro­vide a direct down­load link here. I assume this stuff is cov­ered under Fair Use, but I am going to go the con­ser­v­a­tive route and just tell you to do a Google search.

Plug in your Kindle, and copy the AZW file out of its /Documents folder and into the same folder as your Python scripts. To get the encryp­tion key that Amazon uses to encrypt your eBooks, cd to the direc­tory con­tain­ing the Python scripts and use this com­mand, sub­sti­tut­ing the B12345678 with your actual Kindle ser­ial num­ber:

python ./kindlepid.py B12345678

This should come back with a result along the lines of:

Mobipocked PID for Kindle serial# B12345678 is 1Q4Y2VZ*RE

Save that mix of num­bers, let­ters, and aster­isk. That’s the key that Amazon uses to encrypt all eBooks for your spe­cific Kindle.

Next, use that key to decrypt the book. You’re actu­ally going to have to do this twice. One will work and one will not (and it will be dif­fi­cult to know which one worked until you load it in a viewer in the next step). You will need to run the dedrm v0.01 and dedrm 0.02 com­mands. They dif­fer slightly in the com­pres­sion method used, but the wrong com­pres­sion adds some garbage and bad meta­data to your book. Be sure to replace the 1Q4Y2VZ*RE with your key derived from the above com­mand. Also, leave those single-quotes in there.

python ./mobidedrm.py mybook.azw mybook1.mobi '1Q4Y2VZ*RE'

python ./mobidedrm2.py mybook.azw mybook2.mobi '1Q4Y2VZ*RE'

Either mybook1.mobi or mybook2.mobi should be the cor­rect decrypted ver­sion of your eBook. The next step is to fig­ure out which one is the good one and to con­vert it to ePub.

Convert AZW/MOBI to ePub

Import the first MOBI file into Calibre and use the “View” icon in the tool­bar to see it in the viewer. Verify that this is a non-broken doc­u­ment.

You first want to bring up the table of con­tents by using the icon in the tool­bar on the left. Click around in the table of con­tents to ver­ify that chap­ter mark­ers really do line up with the chap­ter text. Skim through a few dozen pages look­ing for weird HTML or XML frag­ments (angle brack­ets and equal signs that look out of place).

If every­thing looks good, you imported the good doc­u­ment. If things look funky, close the viewer, delete the doc­u­ment from Calibre, then import the other one.

Once you have the good doc­u­ment imported, select it and click on the “Convert E-Books” icon in the tool­bar. Be sure to change the out­put for­mat to EPUB. You can cus­tomize other options as you see fit. Click “OK” to con­vert.

Once con­verted, you can find the file in your Calibre folder or you can use the export func­tion­al­ity to save it to disk.

Drag the result­ing *.epub file into the Books sec­tion of iTunes and you’re set. The book you pur­chased on Amazon can now be read in Apple’s iBooks appli­ca­tion.

Posted in: Books Gadgets

41 thoughts on “Migrating from Kindle to iPad: An Illustrated DRM Primer

  1. I love this post– thank you for tak­ing the time to share it. I am, how­ever, stuck. My kin­dle ebook was down­loaded via the kin­dle app on the iPad, and as I have no Kindle device, I have no ser­ial num­ber that is eas­ily acces­si­ble to me.

    Is there any way to retrieve the nec­es­sary kin­dle ser­ial num­ber used on non-kindle devices?

    1. Sorry, I’m not really famil­iar with how to get files out of the iPad, or even what they use as an encryp­tion key once the file is avail­able. Perhaps if/when a jail­break is released for the iPad, it might be pos­si­ble, but if it’s any­thing like the iPhone, that involves set­ting up a net­work and SSH server and lots of more hard­core unix-y things, whereas you can just copy the *.azw file out of the Kindle as if it’s an exter­nal drive.

      1. well I was able to get the .azw file out just fine by down­load­ing the mac­in­tosh client of the kin­dle reader and whis­per­sync­ing it to my HD. But the dilemma of fig­ur­ing out the key remains. Thanks for your response.

  2. Your instruc­tions are very help­ful. I was able to con­vert a lot of my books into ePub from my Kindle to my iPad. But won­der­ing, what about those titles that have spaces. I keep on get­ting a error msg that says : No such file or direc­tory: “A” because my azw file has a title that begins with “A Fine...” I have tried putting “_”, “-”, ” “, ’ ‘, “\” in place of the space but no dice... Will I be able to con­vert these books? Thanks for your help.

    1. For file­names with spaces on the com­mand line, you have two options. First, you can spec­ify each space as “\ ” (that’s back­slash space). Alternately, you can just wrap the whole thing in single-quotes like ‘A Fine...’ and skip using back­slashes.

      Or you could totally sub­vert it and rename the file to get rid of the spaces — make ‘em under­scores or some­thing. The file­name is just a file­name. The name of the book is a field con­tained inside of the file.

      1. Thanks! I got rid of the spaces and the con­ver­sions were suc­cess­ful!

        I also found that some of my Kindle files have an azw1 exten­sion vs. just plain azw. I have not been able to con­vert them using python. Do I use a dif­fer­ent con­verter?

        1. az1 files are Amazon’s Topaz for­mat (when down­loaded via com­puter they come out with the .tpz exten­sion, and .azw1 when grabbed with WhisperNet directly onto a Kindle). Topaz files are funkier than .azw files when con­vert­ing. I’ve stripped the DRM from all of my Topaz files using the scripts in the “tools_v1.6b” archive (avail­able here & there on the inter­tubes), but instead of just a de-DRM’ed book, it leaves me with an HTML file. Calibre can then con­vert that to epub, but the OCR isn’t so well cor­rected in Topaz files. You might want to edit it a bit before con­vert­ing.

  3. The thang which con­cerns me is the app kindlepid.py is down­loaded from Paraguay and prob­a­bly comes loaded with spy­ware from some cloaked Chinese hacker — I’d rather wait for Apple to give us the app to import kin­dle books into iPad — in the mean­time . . . just got my iPad last week and I love it — only have 6 books from kin­dle for iPhone so not too con­cerned about switch­ing between apps to read. Inconvenient, yes, worth the risk of los­ing any per­sonal info to a hacker with a key­stroke log­ger imbed­ded into a script — NO — per­haps I am overly cau­tious but I am not tak­ing the risk. Great info though, thank you for shar­ing

    1. I wouldn’t hold your breath on an offi­cial import tool. Converting your­self falls, as best as I can tell, in the fuzzy gray area of “Fair Use” under the DMCA. I’m not sure that a big com­pany could pro­vide the same thing — I have seen sev­eral com­pa­nies that tried to pro­vide DVD rip­ping soft­ware that then got sued out of exis­tence.

      As far as mal­ware in the Python script: it is just a script file. It’s about 40 lines of text when you open it in a text edi­tor. If you have any programmer-inclined friends, you can have them take a look at the con­tent. I had sim­i­lar con­cerns when I first down­loaded from a site of sim­i­lar ques­tion­able ori­gins. Fortunately, Python is fairly straight­for­ward and read­able.

    1. It is pos­si­ble and the the­ory and steps are the same as above, but the key would be dif­fer­ent. I don’t know how the PC version’s keys are gen­er­ated, unfor­tu­nately. If you knew the encryption/decryption key for the PC version’s AZW file, you’d be able to do it.

  4. Sorry sioned, your ques­tion only asked if azw files could be con­verted to epub, which is cov­ered here (use cal­i­bre). However, the good news is that what you are try­ing to do can be accom­plished too (I’ve done it).

    Basically you will still use 90% of this guide. When you get to the bits hat require you to use a kin­dle, refer to the link that Castell posted (reply #4). The link will direct you to a guide that shows you how to get your ebook file off your device and onto your com­puter, and then how to get a use­able ser­ial num­ber from iTunes. Once you’ve got that, you can return to Brian’s guide here and resume the process. Hope that helps.

  5. Awesome guys, thanks Taylor and Brian. I will give it a try later today, frankly I’m tired of mess­ing with this stuff for 2 days with min­i­mal results, lol. ;)

  6. Ugh, I’ve been to all the links and I’m still strug­gling.. can u tell i’ve never done any­thing like this before? lol.
    Anyway, here’s my prob­lem, i don’t have an ipod or i-anything.

    I have down­loaded a book I want onto the Kindle for PC app, I have python, I need a Kindle PID to dedrm the .azw file so I can turn it into an .epub file and use it on a dif­fer­ent ereader for­mat.

    All the links i’ve read tell you how to get the Kindle PID if you have an ipod, etc.., which I don’t, so is there any­way to get a PID to use with python to dedrm this book??

    Thanks for all the help, i’m still hope­ful i will get to read what i want the way i want...eventually. ;)

  7. Oh, my apolo­gies Sioned, I have not done it the way you are ask­ing. I think I heard of some­one get­ting it to work, but I don’t remem­ber in detail. I think there is some sort of dummy azw file you down­load, and when you run a script on it, it pro­vides your PID. I would dou­ble check the link posted by castell above for this infor­ma­tion, oth­er­wise I don’t know how to help, sorry =(

  8. Brain Engima or oth­ers that might be able to assist,

    Does any­one know how to con­vert .azw from Amazon sub­scrip­tion news papers into ipad read­able for­mat? I know the kin­dle for ipad app. and kin­dle for mac app works for ama­zon pur­chased books but it does not work for news paper sub­scrip­tions. Anyway, around this?

    thanks,

  9. I’ve used dif­fer­ent ver­sions of mobid­e­drm and all of them give me the fol­low­ing error:

    Error: no key found. maybe the PID is incor­rect

    I’m sure my PID is cor­rect and I’ve checked my Serial# a hun­dred times. I’ve tried both the PID with ’ and with­out. The orig­i­nal book title is Zombie, so there’s no need for ’ there.

    I’ve tried this com­mand line and sev­eral vari­a­tions of it. I’ve also tried it with dif­fer­ent ebooks both trans­fered from my Kindle and down­loaded from Amazon.

    python ./mobidedrm.py Zombie.azw mybook1.mobi ‘XXXXXXX*XX

    I’m run­ning out of ideas. Anyone has any sug­ges­tions?

  10. No mat­ter what I do, I get the mes­sage “Error: no key found. maybe the PID is incor­rect” I have obtained the 40 digit num­ber from both iPad AND iPhone. I used the con­verter found at Kindle Tools PID Generator as well as the “python kindlepid.py XXXXXXXXXXXXXXXX” There is both an older ver­sion which gen­er­ates num­bers with the *XX in them and a newer one which has no * and pro­duces the same num­ber as the PID Generator. I have the Kindle Whispersync appli­ca­tion on my Mac and have the down­loaded files in a folder “My Kindle Content” There, each azw file has a com­pan­ion mbp file. The web says “This file is used to store meta­data like book­marks, anno­ta­tions, last read posi­tion and other data. The MBP file is also used on the Kindle to store infor­ma­tion about a book being read on the device.” Any con­nec­tion to the prob­lem of con­ver­sion? I am stuck — and it seemed so easy.

    1. Do keep in mind that every device or appli­ca­tion that has a copy of the ebook has a dif­fer­ent key. The file down­loaded over WiFi or 3G into your iPad has a dif­fer­ent key than the one down­loaded over WiFi or 3G on your iPhone, which has a dif­fer­ent key than the one down­loaded by the Kindle app on the Mac. If you’re try­ing to use the iPhone or iPad key on the desk­top software’s file, it won’t work. You’ll need the key of the desk­top soft­ware. I’m not famil­iar enough with the soft­ware to know what the key might be or how to find out what it is (I mainly down­load onto my hard­ware Kindle and use its ser­ial num­ber, so haven’t had the need to inves­ti­gate fur­ther).

    2. One thing to keep in mind is that with the Kindle 2.5 update a month or two ago, they changed their DRM. Apparently now it uses a sep­a­rate key per-book instead of a sin­gle DRM key on a given device (per-PID). I’m not sure how that effects files down­loaded onto the iPad/iPhone, but files down­loaded on my Mac for read­ing on my Kindle don’t de-DRM eas­ily any­more using the scripts.

      What I’m doing to work around this is use VMWare to run Windows on my Mac (yeah I know, I know), and run the Windows ver­sion of the Kindle app there. It still uses the old DRM scheme so the PID and such work as before. I’ve also turned off auto-update on it, so it hope­fully won’t update itself to the new DRM scheme.

  11. @Snorkledorf: How did you find a PID for the Mac itself, as the iPhone or iPad ones will not work with the files down­loaded on my Mac? Finding that would be help­ful.

    1. Well that’s the thing, I’ve never actu­ally fig­ured out how to strip the DRM for books that are down­loaded via/for the Mac ver­sion of Kindle reader. The PC ver­sion DRM is a known quan­tity how­ever, so for me it’s worth it to fire up Windows in a vir­tual machine on the Mac, then do all the de-DRM’ing inside Windows, using Kindle Reader for PC.

      Since any Mac can run Windows (I use VMware Fusion, but Parallels or Boot Camp will work just as well), that was just the eas­ier route for me. Even if I have to grit my teeth for the dura­tion. :)

  12. If any­one is get­ting this error:

    Error: no key found. maybe the PID is incor­rect

    I have one solu­tion... In my case, I was using low­er­case let­ters for the ser­ial num­ber. WRONG! The kindlepid.py script is CASE-sensitive.
    (Yes, cAsE shouldn’t mat­ter with a hex num­ber, but in this instance it does. Hopefully a new ver­sion may rec­tify the bug! I’d actu­ally like to see the decrypt script process an entire direc­tory at once...)

    Everything decrypted just fine after using an all-caps ser­ial num­ber! (Except for the book we bought after the firmware v2.5 upgrade...)

    Hope this helps some­one Googling for an answer...

  13. Thank you soooo much works great on my MAC, and I don’t have a kin­dle but can still get files from ama­zon con­verted to read on my Kobo....I tried all the other stuff with only a headache, but yours was sim­ple and easy.

  14. If you’re try­ing to use the iPhone or iPad key on the desk­top software’s file, it won’t work. You’ll need the key of the desk­top soft­ware. I’m not famil­iar enough with the soft­ware to know what the key might be or how to find out what it is (I mainly down­load onto my hard­ware Kindle and use its ser­ial num­ber, so haven’t had the need to inves­ti­gate fur­ther).

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>