Upgrade from iPhone v1 to 3G S - now I have no Collections

[Canuck]

Hi folks, I just completed the migration of all my apps, documents and data from my vintage "gen 1" iPhone (8GB) to the latest iPhone 3G S. I've seen all the major apps' data migrate over successfully, except for PocketPedia. Unfortunately, I'm still one of those silly buggers that uses a Windows PC for my main/work computing - my iPhone is the only ray of sunshine in an otherwise dark silicon existence, so I don't have any Mac-based backup from my old iPhone.

I've searched through these forums and seen a few hints (e.g. here: http://www.bruji.com/forum/viewtopic.ph ... 60&start=0) on how I might be able to manually retrieve the file from the old phone (hasn't been wiped yet - thank gods!) and drop it into the right location on my new phone. At the very least, I'd love to make sure I get one good backup of the data file from my old iPhone, so I can safely wipe it and give it to my girlfriend (who's suffering harshly with a crappy Nokia).

I have file-manipulating utilities on both phones including "Mover" and "FileAid", and I'm willing to get additional ones that might give me the ability to see the PocketPedia mysql file(s) and copy them to another place on my iPhone (at which point FileAid should let me get the file off the phone). Heck, at this point, with 100's of entries in my Collections, I'd be willing to *pay* a few bucks if I could find an app that would assist me here.

I'll probably unlock the old iPhone, so if there's some PC-based "hack" tools I need to get and use, that's also a possibility. However, given the 3GS is brand-new, I would be surprised if there was already a "hacking" tool that worked against Steve's latest anti-consumer workarounds.

Heck, I'll even do some digging in the %APPDATA%\Apple Computer\MobileSync\Backup\ files (where there are thousands of randomly-named *.mddata and *.mdinfo file - presumably backups of individual files off the iPhone), if you can tell me what sorts of strings to GREP for.

Thanks for any tips you might be able to offer.

[Canuck]

I've done some more digging and haven't come up with a solution yet, but wanted to share what I've tried (so others don't go down the same blind alleys - OR others may be able to fill in the remaining steps):

• This article http://iphonefreakz.com/2008/04/24/mana ... lewindows/outlines the use of Total Commander and a plug-in called "T-PoT" to explore the user-accessible partition on the iPhone's flash drive. I explored most of the folders but couldn't find anything that had the .SQL extension, or that looked like a directory that would've been created for use by Pocketpedia.
• This forum post http://www.bruji.com/forum/viewtopic.ph ... 97&start=0 mentions the SQLite database file is named pokectpedia.sql, and that the .mdbackup extractor should be able to grab a copy of the file from the iPhone. No, wait - this thread here http://groups.google.com/group/iphonewe ... 50be?pli=1 clues Mike in to the idea that the .mdbackup files (the backup files cached on my PC when iTunes does that lovely, huge long backup of the iPhone) each store one individual file off the iPhone.
• This article http://forums.macrumors.com/archive/ind ... 61967.html lists three apps (online, Windows, Mac) that assist with extracting data from the .mdbackup files.

[Canuck]

Oh baby, I think I got it!

Here's the process I'm following:

• First, I already had a full backup from the old iPhone on my PC - if you don't already have one, make sure to sync the old phone with iTunes (don't worry, it'll keep multiple phones' backups and configurations separate) and let it perform a FULL backup.
• Once the backup is complete, you'll find the hundreds or (for me at least) thousands of .mdbackup and .mdinfo files in your user profile under %APPDATA%\Apple Computer\MobileSync\Backup. Each iPhone will have a separate folder under that \Backup directory. For me, it was easy to see which folder was for which phone - 425 files in one folder, 5840 files in the other. I'm *pretty* sure the new iPhone I got yesterday is associated with the former.
• I downloaded the free version of reincubate's iPhone Backup Extractor from here: http://www.reincubate.com/labs/iphone-b ... p-windows/. There's only two files, so just drop it pretty much anywhere and fire up iPhoneBackupExtractor.exe.
• It looks in your Windows user profile for the default iPhone backup folder (i.e. the one I noted above), and will let you select among the different backups it finds. It shows you the date when the backup was taken, how many files are there and the Serial # and IMEI of the device that was backed up. Even if you somehow choose the wrong backup, no big deal - just hit the Back button or restart the app.
• Choose the option "Restore some or all files from backup manually" and then browse the tree view to find the pokectpedia.sql file listed under Application > com.bruji.pocketpedia > Library > Application Support > Pocketpedia
• check the checkbox next to pokectpedia.sql, tell it where to extract the file to, wait for the polite delay, and hit Finish.

There, that's the scariest step completed - I now have a backup of the Pocketpedia data that I so lovingly compiled over the past year or so.

Next, I decided to figure out which .mdbackup file actually stores that data, so that I (might) experiment with dropping that file into the backup directory of my new phone, and see if I could restore that file (along with the rest of the backup) to my phone via iTunes.

So I performed the sequence once again while running ProcMon (from Sysinternals: ) in the background, watching for filesystem activity for the iPhoneBackupExtractor.exe app. The only files that it touched in the MobileSync\Backup\ area were:

• 2253723fa6d690496f9dbf28aa623daa5a0fb495.mdinfo
• 2253723fa6d690496f9dbf28aa623daa5a0fb495.mddata

And I noticed a telltale sequence of ReadFile() and WriteFile() calls that seem to prove that the .MDDATA file is the one being read from when iPhoneBackupExtractor is creating the backup file for pokectpedia.sql.

Next experiment to try then, is to:

• Create a dummy entry in PocketPedia on the new iPhone (so that Pocketpedia creates a pokectpedia.sql file that can be backed up by iTunes)
• get a full backup of the new iPhone (so that I can see if the .MDDATA file on the new phone is named the same as the one on the old iPhone, and so that if I perform a restore it'll have all my current data and settings, and not the ones from the old backup I got yesterday)
• copy the old iPhone's .MDDATA file from the last backup to the new iPhone's backup folder (thus overwriting the new, dummy file with the old one)
• restore the fresh, full backup to my new iPhone
• fire up Pocketpedia and (hopefully) have a very wonderful moment of data transfer

Conor, do you happen to know if:

• the .MDDATA filename I've quoted above is universal across all iPhones,
• there's any device-specific encryption of the .MDDATA file that would prevent me from uploading one iPhone's backup file to another iPhone, and
• there's any way that's more efficient/elegant than what I've outlined, to upload the backup from one iPhone to another iPhone?

This sure seems promising, and I know it'll make me real happy to be able to migrate from one iPhone to another this one time, but I know that it'd be a real pain for people to do this repeatedly every week or month as a way to "sync" their Collections between multiple iPhones. (Sorry to you folks that are trying to fulfil that latter scenario - this may not be quite what you're after.)

[Conor]

That is a thorough research and explanation, and unfortunately all correct. Making it complicated to backup your iPhone data. Not having a Windows machine I am not very informed with the windows tools. The most important file is the pokectpedia.sql, once you have it on your hard drive test it out with SQLite3 to make sure it's a complete file. You should be able to open up the command line and use SQLite3 pathtothefile/pokectpedia.sql and then type ".dump" and there should be a lot of data that sort of resembles your pocketpedia data (on the mac SQLite3 is available from the command line not sure if it comes pre-installed as a binary on Windows).

2253723fa6d690496f9dbf28aa623daa5a0fb495 is a random number that gets generated for each app when it's downloaded, for the sandboxing process that the iPhone performs. There should be no device specific encryption, in fact when you plugged in your new iPhone it should have asked if you wanted to restore a copy of your previous iPhone from the backup.

The above process is the best way to go without a Mac. It has the disadvantage that the covers are not included. So you will have to click on each cover in the details pane of the new iPhone to have Pocketpedia download a new cover and this can be time consuming for a large number of entries (although hopefully all the 2253723fa6d690496f9dbf28aa623daa5a0fb495 files include the covers as well, that should be easy to verify with the size of the file, anything over a mega would be including the covers). If you can find access to a friend's or work Mac, you can use our Pedia programs in their trial version to both extract the Pocketpedia SQL file as well as to import a pocketpedia SQL. Hold own the option key on a Mac and select "Pocketpedia" from under the main help menu. This will bring up a debug window with backup and restore options. It also has a button for import to put the entries from Pocketpedia into the Pedia (you would have to use each specific Pedia; e.i. Bookpedia will only import books). Which would allow you to download more information in the Pedias and then sync them back, but you would have separate collections and would need the upcoming Pocketpedia version to be able to continue to add to those separate collections instead of the Pocketpedia collection.

[Canuck]

Thanks for your very reassuring and informative response Conor!

Here's more detail in my quest to migrate PocketPedia from my iPhone 2G to iPhone 3G S:

• The backup file that ProcMon flagged when backing up the pokectpedia.sql file from my new iPhone had exactly the same name as the one flagged from the old iPhone's backup. Perhaps 2253723fa6d690496f9dbf28aa623daa5a0fb495.MDDATA is the filename on every iPhone? (Or perhaps the randomness generator is seeded by something in the copy of iTunes that's being used?)
• The matching .MDINFO file from each backup was characterized as "identical" by WinDiff.exe
• Yes, the first time I plugged in my new iPhone to my PC, iTunes asked if I wanted to restore a backup - I'd chosen yes, and pretty much everything except PocketPedia survived the transfer without incident.
• The pokectpedia.sql file from my old iPhone was approx. 670 KB; the same file from my new iPhone (with the one dummy entry, to ensure the file got created) was around 7 KB. The iPhoneBackupExtractor showed me that there were folders (Covers and Thumbnails) full of files for both the thumbnails and the full-sized covers; the file sizes for the extracted pokectpedia.sql confirm that the covers & thumbnails couldn't possibly be stored in the .sql file.

Unfortunately, all this work aside, it appears that there's still some step missing from my sneaky approach to migrating the Pocketpedia collections data from one iPhone to another via the PC/iTunes/iPhoneBackupExtractor combination. After carefully copying the 670 KB backup file to the Backup folder for the new iPhone (having renamed the original backup file in the new Backup folder so that it was still available if need be), and waiting patiently through a full Restore from that backup (initiated via iTunes), I fired up PocketPedia on the newly-restored iPhone and discovered that the Collections were not only not sync'ed with my old Phone's collections, but that I was back to a completely blank Collections list (i.e. even the dummy Collection I'd created - and then tried to Restore over top of - was no longer present).

It seems there's something preventing either (a) the backup file from getting restored to the iPhone, or (b) Pocketpedia from recognizing and opening a pokectpedia.sql file that's been restored to the iPhone. (The latter would at least be consistent with the behaviour of Restoring the full backup from the old iPhone to the new one, when I'd first unboxed the new iPhone.)

Here's the final tidbit: after the full Restore (which should've included the 670 KB pokectpedia.sql backup file I'd added to the backup folder), I performed a full backup of the new iPhone - to see whether the pokectpedia.sql file is still present on the iPhone, but is just not able to be read by Pocketpedia. Then I fired up iPhoneBackupExtractor.exe once again, and once again performed an extraction of the pokectpedia.sql file from the new iPhone. (If my backup/restore trick worked, then it should be 670 KB; if the backup of the file from my old iPhone never successfully "took" on the new iPhone, then the file should be 7 KB or less in size.)

The result? The extracted pokectpedia.sql file was 5 KB in size. This leads me to believe one of three things is true:

• (1) There's something 'wrong' with the 2253723fa6d690496f9dbf28aa623daa5a0fb495.MDDATA file that iTunes recognizes during the Restore process, and the file never gets uploaded to the iPhone.
• (2) There's some integrity checker running in the iPhone that checks the 670 KB file after it's been restored to the iPhone, and the iPhone decides the best course of action is to delete the file before it's even seen by Pocketpedia.
• (3) There's something 'wrong' with the pokectpedia.sql file that's been restored to the new iPhone, and when I run Pocketpedia on my new iPhone, it decides there's an issue with the file for which it should delete the file and start fresh.

Conor, can you confirm whether you have any such logic in the Pocketpedia app that resembles alternative #3? If not, then my next step is to look in the backup folder for any metadata files that might keep some hash/CRC (or even filename/date record) for the 2253723fa6d690496f9dbf28aa623daa5a0fb495.MDDATA file, and see if I can successfully manipulate that. I don't have ready access to a Mac (I work with a bunch of PC geeks, and I don't own any myself - but I guess that's obvious), so pursuing the Bookpedia approach isn't my lowest-energy option quite yet. (Not to mention I'd like to figure out a solution that can be used by other PC users.)

[Conor]

I have no code that would replace the file. If the file was present but not readable by Pocketpedia it would go crazy. Here is the exact code:

Code: Select all

 NSString *writableDBPath = [self sqlPath];
    success = [fileManager fileExistsAtPath:writableDBPath];
    if (success) return;
    // The writable database does not exist, so copy the default to the appropriate location.
    NSString *defaultDBPath = [[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:@"DefaultDatabase.sql"];
    success = [fileManager copyItemAtPath:defaultDBPath toPath:writableDBPath error:&error];
    if (!success) {
        NSAssert1(0, @"Failed to create writable database file with message '%@'.", [error localizedDescription]);
    }

As you can see if a file exists at all then it's never replaced. On SQLite trying read and write to an incomplete file there would be numerous SQLite errors and Pocketpedia would most likely crash and the file would not be changed.

There could be some kind of permissions on the file that are not set by the restore that cause Pocketpedia not to see the new file. I am baffled as why the original restore would not have brought the database along. I have just tried two complete restores of the iPhone using 3.0 with different Pocketpedia databases and both have come through (Developing it's common for me to have to restore the iPhone between OS versions).

I truly appreciate all the work and time you are putting into this to help out other PC users that might have an issue with the OS upgrade as well. If you don't mind please send me the pokectpedia.sql file so that I can verify it for you that it's as it should be and that Pocketpedia should have no trouble reading it.

[Canuck]

If you can find access to a friend's or work Mac, you can use our Pedia programs in their trial version to both extract the Pocketpedia SQL file as well as to import a pocketpedia SQL. Hold own the option key on a Mac and select "Pocketpedia" from under the main help menu. This will bring up a debug window with backup and restore options. It also has a button for import to put the entries from Pocketpedia into the Pedia (you would have to use each specific Pedia; e.i. Bookpedia will only import books). Which would allow you to download more information in the Pedias and then sync them back, but you would have separate collections and would need the upcoming Pocketpedia version to be able to continue to add to those separate collections instead of the Pocketpedia collection.

I was never able to resolve this until just this week, when I bought myself a spankin' new Mac Mini. Now I've got the trial version of Bookpedia installed, I've found the debug menu with "Import", "Backup" and "Restore" buttons. I've clicked "Restore", then selected the pokectpedia.sql file that I've preserved so loviingly all along, and then I'm presented with a screen that mystifies me:

"Initiate sync
Please press the sync button in Pocketpedia and select Bookpedia if prompted."

I've fired up Pocketpedia on my iPhone and looked for anything resembling a sync button, but nothing there. What am I missing? Please help - I feel like I'm so close to getting all this back, and then I'll be a smilin' man once more. Thanks for any help you can provide, Mike

[Nora]

The 'Sync' button should appear on your Collections screen in Pocketpedia. When you run Pocketpedia, you'll see three different screen options: Collections, Amazon and Results.
For the sync button to appear, make sure that you have Bookpedia running on your computer, that your computer and the iPhone are connected to the same wireless network and that your Firewall is either turned off or that Bookpedia is added to the list of 'allowed' applications. You might want to take a look at[=http://www.bruji.com/videos/pocketpedia/sync.htmlurl] this little video[/url] for some troubleshooting tips on the synching process.

Welcome to the the wonderful world of Mac!

[Canuck]

Thanks for your response. I tried the sequences you described, but no matter how many times I tried to get the Sync button in Pocketpedia, it wouldn't show up.

I watched the troubleshooting video and noticed (a) my firewall isn't even enabled, (b) there are some icons in the Pocketpedia version in the video that I'm not able to see in my version of Pocketpedia (the one installed on my iPhone). I began to wonder if I had an outdated or defective install of Pocketpedia, so I decided to uninstall and reinstall Pocketpedia (which has become a pretty standard response for many iPhone app issues).

Unfortunately, it looks like Pocketpedia isn't currently listed in the App Store - I tried searching on "pocketpedia", "pocket", "pedia" and "bruji", but nowhere could I find the Pocketpedia I've installed a few times before.

Did something happen? Are you about to launch a new version, and Apple removes the old one just before making the new one available? Or is there something more serious going on?

[Nora]

Pocketpedia isn't available at the moment because Amazon obliged us to remove the application from the App Store a few months ago. You can read more about that here and here.
But we're working on a new version right now which will hopefully be ready for beta testing real soon.

[Canuck]

Pocketpedia isn't available at the moment because Amazon obliged us to remove the application from the App Store a few months ago. You can read more about that here and here.
But we're working on a new version right now which will hopefully be ready for beta testing real soon.

Whoops! Well that's my bad for not keeping up on the forums here. Oh well.

At this point, then, is it likely that there will be a way for me to re-use the data contained in the pokectpedia.sql database file I've got? I.E. will the new pocketpedia beta be able to use that data (presumably via the same two-way sync I was trying to use with the Bookpedia <> pocketpedia combination yesterday)? Or am I better to just start over from scratch?

And if I'm better to start over from scratch, should I just wait for the pocketpedia beta, or should I start off building a catalog of my books with the Bookpedia trial I've now got installed? [i.e. does the Bookpedia trial give me a "head start" in building up the catalog for the new Pocketpedia, or is Bookpedia only a viable sync source when it's the fully licensed version?]

Thanks again, and my personal condolences on the Amazon dust-up.

[Nora]

The best would be to start from scratch in Bookpedia. But send us your pokcetpedia.sql file and we can create a regular Database.pediadata file (the Pedia database file) out of that for you here so you'll get a head start. You'll find our email address on our support page.