Sunday, March 24, 2013

Virtual SD card on Android


Since Android Honeycomb, Google changed the way to manage internal memory on Android devices. Before Honeycomb, every user had one separate partition on his device called userdata (/data), where he could install applications and where all user settings were stored (home screens, applications data, contacts, and all the rest you loose after doing so called "full wipe" on your device). Apart of userdata partition, all Android devices had microSD card slot to save pictures, movies, backups etc.. Now it looks completely different, but let's start from the beginning. There are several approaches to this subject, I'll present here all those I am aware of.


  • userdata partition + microSD card

This is the mentioned above pre-Honeycomb approach. There is userdata partition, where you can install all your applications and you also have a possibility to insert microSD card. Nothing more to explain. Only devices running Android Gingerbread and older versions have such configuration, so it's getting less and less popular.


  • userdata partition + virtual SD card on userdata partition

This is the new approach presented for the first time in Honeycomb. Instead of having /data partition together with expandable microSD card slot of any capacity, Google decided for something different. Instead, /data partition became very large (16/32/64 GB) and inside you can find /data/media folder that contains all the files you can see as your SD card content. How does it work? Without too much technical explanations, there is so called fuse tool which emulates media folder inside userdata partition as a separate storage device. As a result, after connecting smartphone to the PC you can browse the content of /data/media location, so if it was a microSD card. The biggest downside of such approach is a high risk of loosing all your virtual SD card content in case of any serious /data partition failure. Also, such partition can't be formatted with mkfs.ext4 without loosing content of virtual SD card, because you can't format device partition just partially. You can use e2fsck tool to check for potential errors, but sometimes partition format is the only way out. How does "full wipe" work then? Well, it's a little bit complicated. First of all, you can't format mounted (in use) partition. You need to unmount it first. Once unmounted userdata partition, you can't flash any ZIP file from inside recovery, because ZIPs files are stored on virtual SD card (/data/media) and remember that userdata partition is currently unmounted, because we want to format it. There is a workaround for it - you can run mkfs.ext4 from inside /cache partition or you can use command prompt. Now, what if you need to remove the whole content of your userdata partition, but you want to keep virtual SD card content at the same time? There is a workaround for this as well, but instead of formatting entire partition, you need to remove all files excluding /data/media location. Example:

#!/tmp/bash
# Remove content of /data partition excluding data/media files
cd /data
FILES=(*)
for i in *; do
if [ "$i" != "media" ]
then rm -R "$i"
fi
done


This way you can sort of wipe userdata, but it doesn't format the partition, so you can't fix broken file-system with it.  Why this point is the longest one? Because it took me quite a few words to explain the relation between virtual SD card and media folder on userdata partition (/data/media). So basically, what you read here applies to every configuration with virtual SD card emulated on userdata partition.


  • userdata partition + virtual SD card on a separate partition

This approach is not very popular, and it's a shame because it seems to be much more practical rather than the previous one. Instead of emulating SD card from userdata partition, there is a separate, large partition with vFAT file-system. That means you can format your userdata partition anytime you want without loosing content of your virtual SD card, or from inside custom ROM, because userdata can be freely unmounted. The only device I've seen so far with this approach was HTC One X.


  • userdata partition + virtual SD card on userdata partition + microSD card

This seems to be the most desirable solution for many Android users. It works similar to approach described in the second point, so everything I wrote about /data/media is valid here as well. However, every user have the ability to insert extra microSD card inside his device, so he can easily backup virtual SD card to microSD card or format userdata partition without loosing all pictures etc. (if previously stored on microSD card). This is the most common configuration for Samsung devices. But having removable microSD card is not only an advantage. First of all, any kind of microSD card (even SDHC) will be always slower than internal eMMC memory. It depends on many factors like card speed (class 2, 4, 6, 8 or even 10), on-board controller, I/O scheduler and more. Secondly, microSD card damage risk is higher then damage risk of internal eMMC memory. Out of question is the benefit to expand the memory with 64 GB microSD card, but it's definitely the minority of power users, who are buying large capacity cards. For the vast majority of users, internal memory with 32 GB capacity is more then enough to store their favorite music or pictures.


  • userdata partition + virtual SD card on a separate partition + microSD card

This approach is theoretically possible, but personally I've never seen device with such combination. For me, this is the best combination. You have possibility to use external microSD card and virtual SD card is not a part of userdata partition, but it has it's own, separate vFat partition. Such configuration gives you control over all your data and possibility to manage it however you want.

Do you have any questions or want to share some opinion? Please leave a comment below! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) down this post!



27 comments:

  1. I think we go in the same direction as you have written in the previous article about removable batteries.

    As long as there are "moving" parts such as external SD cards as well as changeable batteries you will never ever have the possibility to build compact devices such as the ONE. This is the prize we have to pay for devices like this.

    Just beware that unlocking your device with HTCDEV has affect to the virtual SD card. This might be a disadvantage towards the external SD card but I think we can (or better devs can) handle this.

    CHeers

    K.

    ReplyDelete
    Replies
    1. Yes mate, but this time I tried to be more careful with expressing my preferences ;) Personally I don't need microSD card in my device. Possibility to build compact device with uni-body design is more important to me. But I understand if people want microSD due to lot of files they want to carry on with them.

      Delete
    2. I'm curious.. what is the effect that user konsti23 says unlocking our devices with HTCDev has on the virtual SD card? thank you, and I'd also like to know what is the configuration on the One X+

      Delete
  2. What is the configuration on HTC One

    ReplyDelete
    Replies
    1. userdata partition + virtual SD card on userdata partition

      Delete
  3. How about the zte avid 4g......i could use some help

    ReplyDelete
    Replies
    1. No idea, I've never had this device and never seen its software.

      Delete
  4. how can i program the android 4.x.x OS to use userdata partition + virtual SD card on a separate partition + microSD card? i have a galaxy tab 2 7.0 p3113 and it has 8 GB internal memory, 4GB used for system and the other 4 for data and apps, i want to use an external 32 gig card as that second 4 gb partition, and use the hole 8 gb for the system.

    ReplyDelete
  5. Hello! I needed factory reset on my htc one s and I lost my photos. I heard about DiskDigger which can restore my photos from the old partition,but I need to root my device. Before that I have to unlock bootloader. I tried to root with locked bootloader and didn't work. So I have to unlock it. Now the problem is that unlocking bootloader is like factory reset. After unlocking bootloader I will be able to acces the old partition to recover photos? Or is an easier way to recover that partition?

    ReplyDelete
    Replies
    1. HTC One S has a separate SD card partition or it's part of /data partition?

      Delete

  6. I have learn several good stuff here. Definitely worth bookmarking for revisiting.

    ReplyDelete

  7. It’s really a great and useful piece of information.

    ReplyDelete
  8. I am glad that you shared this useful information with us. Pl

    ReplyDelete
  9. Happy I read this best blog

    ReplyDelete
  10. Hi ! this nice article you shared

    ReplyDelete
  11. Great info. Keep on sharing

    ReplyDelete
  12. Thank you for the good story. Considered a great and very useful content.

    ReplyDelete
  13. Wow, superb blog layout! Magnificent, let alone the content fantastic.

    ReplyDelete
  14. I cannot thank you enough for the article post. Great.

    ReplyDelete
  15. I extremely like this post.Thanks for the detailed you provide.

    ReplyDelete
  16. For certain I will review out more posts. Keep on sharing dude.

    ReplyDelete
  17. Really appreciate you sharing this article. Much thanks again.

    ReplyDelete
  18. I’ll be coming back to your website for more soon.

    ReplyDelete
  19. Everything is very open with a really clear explanation of the challenges.

    ReplyDelete
  20. You put truly extremely accommodating data. Keep it up.

    ReplyDelete
  21. thank you for sharing such a great bog it is very helpful.

    ReplyDelete