Is Google blocking apps writing to SD cards ?
Ok, so the subject is a little misleading. A more complete question would be:
At the moment, it does appear so, but it may also be a bug or manufacturers incorrectly implementing a "feature". First, let me clarify the question, for starters to explain which storage I actually mean.
Various Android devices released the past few years have come with "internal" flash storage, as well as having the capability to extend storage by using SD cards.
In Android terms, that "internal" flash storage is also called "external storage". It does not matter that the storage is physically inside the device, it is outside the normal app storage location.
Unfortunately, even though having these two types of external storage has been fairly common, Android still does not provide proper access to these cards through the API. You may think this article as about that - it isn't, this goes a bit deeper.
So, if you have purchased an Android device that came preloaded with 16gb (or 8, or 32, etc) storage and it can also use an SD card, your device has two external storages already.
The internal flash memory would usually be considered the primary external storage location, while the SD card would be considered the secondary external storage location (if internal flash is present). A USB stick or harddrive you can potentially connect would also be secondary external storage.
It appears that in newer Android builds, Google is making it impossible for third party apps (apps that were not preloaded on the device, but you manually installed or downloaded from Android Market / Google Play) to gain write access to the external SD card.
This would make the SD card (and USB devices) completely useless as anything other than read-only storage. Unless you manually copy files there using the built-in file explorer (if your device even comes with one), that is.
I ran into this issue earlier today when working on photo-backup functionality for DSLR Controller and testing it on a Samsung Galaxy Tab 7.7 running Android 3.2. After some time researching the problem, it appears there are a number of other devices also suffering from this "feature" - if you know where to look, complaints about this are abundant. In case of the DSLR Controller example, writing to the internal flash only instead of to SD cards makes the backup feature essentially useless, as will be the case for many other apps and their features. This problem was not present in Android 2.x, though I'm not currently sure in exactly which 3.x build it was introduced.
In the past, an app would request the "WRITE_EXTERNAL_STORAGE" permission, which would grant write access to all external storages (user/group "sdcard_rw"). This has apparently been changed to only grant write access to the primary external storage. A second permission has been introduced called "WRITE_MEDIA_STORAGE", which would grant access to the other external storages (user/group "media_rw").
The problem is, a third party will not actually be granted this permission, only system apps and apps provided by the device manufacturer will normally be granted this permission. There are exceptions, apparently on some devices third party apps will be granted this permission, but according to the AOSP sources, they're certainly not supposed to.
So, what is going on here? Is this a bug? Is this a feature? Is this something meant to be used differently but wrongly applied in firmwares by the OEM? I would sure like to know!
Sources
Of course, you're not expected to just assume whatever I say about this to be correct. Here are some pastes from ICS sources.
system/core/include/private/android_filesystem_config.h:
This shows the AID_MEDIA_RW definition, the group an app must have to have access to secondary external storages.
frameworks/base/core/res/AndroidManifest.xml:
This shows how the "WRITE_MEDIA_STORAGE" permission has "signatureOrSystem" protection. This means it will only be granted to system apps, or apps signed with the same key as the platform (apps provided by the OEM). Note the @hide marker. The description mentions internal media storage, which may hint at external SD cards being marked this way actually being a bug, instead of a feature.
frameworks/base/data/etc/platform.xml:
This show that the WRITE_MEDIA_STORAGE permission grants the "media_rw" group to an app.
system/vold/Volume.cpp @ Volume::mountVol():
This shows that whichever external storage's mount point is equal to the "EXTERNAL_STORAGE" environment variable will get the normal AID_SDCARD_RW group, and apps can get access to this storage with the "WRITE_EXTERNAL_STORAGE" permission. Any other external storage (this includes USB sticks and whatnot) will get the AID_MEDIA_RW group, and thus cannot be written to by third party apps.
The comments are somewhat confusing, as they do mention "primary SD card". The EXTERNAL_STORAGE environment variable often refers to the internal flash instead of the primary SD card. So again, maybe this is not what was intended to happen.
It is not uncommon (but also not a standard) for the internal flash to be mounted in one place and referred to as "the" external storage, with all other external storages mounted in subfolders of that mountpoint. This is a popular OEM workaround to the problem of the Android API not properly supporting multiple external storages (and Google seems to be refusing to fix this situation). This way, a third party app only needs to know "the" external storage location, and due to the subfolder structure, it can still access all of the external storages. This specific way of doing things obviously causes problems with the code posted above.
Hopefully, somebody at Google can shed some light on this situation.
Update #1
This has been discussed on Android Developers' weekly Google Plus hangout, as well as some other chats with Google engineers.
None of them had any idea what is going on here - yet. But we have gotten a promise that this specific issue will be looked into, and they will get back to me on this - I hope it will be in dialog!
I also wanted to share with you that even devices like the SGS2 (international) suffer from this issue on ICS, though Samsung has worked around it using a very ugly permission hack. The problem is still there, even if it doesn't show up.
Update #2
By request, here's the way Samsung has worked around the problem in their ICS builds for the SGS2:
/system/etc/permissions/platform.xml:
Well, this is certainly an interesting way to go about working around the problem. What is happening here is that if you ask for the "WRITE_EXTERNAL_STORAGE" permission, you also get the "WRITE_MEDIA_STORAGE" permission - at no extra charge! It should be obvious that this is not in any way a proper solution to the issue.
What makes this even more interesting is that this means at least Samsung engineers are apparently well aware of the problem.
Still looking forward to hear what Google has to say about all this. It worries me that they added code that - based on the code comments - is meant to restrict even more access, while not providing any clear reason why this should happen.
Update #3
An observant commenter on the XDA portal article stated that this issue is patched in CyanogenMod:
That's nice for CyanogenMod users, but doesn't help stock users. The question remains - WHY ?! There doesn't seem to be any reason for this stuff, and it's causing real-world issues to real-world devices for real-world people!
"Is Google moving towards preventing third party apps from writing to secondary external storage ?"
At the moment, it does appear so, but it may also be a bug or manufacturers incorrectly implementing a "feature". First, let me clarify the question, for starters to explain which storage I actually mean.
Various Android devices released the past few years have come with "internal" flash storage, as well as having the capability to extend storage by using SD cards.
In Android terms, that "internal" flash storage is also called "external storage". It does not matter that the storage is physically inside the device, it is outside the normal app storage location.
Unfortunately, even though having these two types of external storage has been fairly common, Android still does not provide proper access to these cards through the API. You may think this article as about that - it isn't, this goes a bit deeper.
So, if you have purchased an Android device that came preloaded with 16gb (or 8, or 32, etc) storage and it can also use an SD card, your device has two external storages already.
The internal flash memory would usually be considered the primary external storage location, while the SD card would be considered the secondary external storage location (if internal flash is present). A USB stick or harddrive you can potentially connect would also be secondary external storage.
It appears that in newer Android builds, Google is making it impossible for third party apps (apps that were not preloaded on the device, but you manually installed or downloaded from Android Market / Google Play) to gain write access to the external SD card.
This would make the SD card (and USB devices) completely useless as anything other than read-only storage. Unless you manually copy files there using the built-in file explorer (if your device even comes with one), that is.
I ran into this issue earlier today when working on photo-backup functionality for DSLR Controller and testing it on a Samsung Galaxy Tab 7.7 running Android 3.2. After some time researching the problem, it appears there are a number of other devices also suffering from this "feature" - if you know where to look, complaints about this are abundant. In case of the DSLR Controller example, writing to the internal flash only instead of to SD cards makes the backup feature essentially useless, as will be the case for many other apps and their features. This problem was not present in Android 2.x, though I'm not currently sure in exactly which 3.x build it was introduced.
In the past, an app would request the "WRITE_EXTERNAL_STORAGE" permission, which would grant write access to all external storages (user/group "sdcard_rw"). This has apparently been changed to only grant write access to the primary external storage. A second permission has been introduced called "WRITE_MEDIA_STORAGE", which would grant access to the other external storages (user/group "media_rw").
The problem is, a third party will not actually be granted this permission, only system apps and apps provided by the device manufacturer will normally be granted this permission. There are exceptions, apparently on some devices third party apps will be granted this permission, but according to the AOSP sources, they're certainly not supposed to.
So, what is going on here? Is this a bug? Is this a feature? Is this something meant to be used differently but wrongly applied in firmwares by the OEM? I would sure like to know!
Sources
Of course, you're not expected to just assume whatever I say about this to be correct. Here are some pastes from ICS sources.
system/core/include/private/android_filesystem_config.h:
Code
#
1
{ "media_rw", AID_MEDIA_RW, },
This shows the AID_MEDIA_RW definition, the group an app must have to have access to secondary external storages.
frameworks/base/core/res/AndroidManifest.xml:
Code
#
1
<!-- Allows an application to write to internal media storage
2
@hide -->
3
<permission android:name="android.permission.WRITE_MEDIA_STORAGE"
4
android:permissionGroup="android.permission-group.STORAGE"
5
android:label="@string/permlab_mediaStorageWrite"
6
android:description="@string/permdesc_mediaStorageWrite"
7
android:protectionLevel="signatureOrSystem" />
This shows how the "WRITE_MEDIA_STORAGE" permission has "signatureOrSystem" protection. This means it will only be granted to system apps, or apps signed with the same key as the platform (apps provided by the OEM). Note the @hide marker. The description mentions internal media storage, which may hint at external SD cards being marked this way actually being a bug, instead of a feature.
frameworks/base/data/etc/platform.xml:
Code
#
1
<permission name="android.permission.WRITE_MEDIA_STORAGE" >
2
<group gid="media_rw" />
3
</permission>
This show that the WRITE_MEDIA_STORAGE permission grants the "media_rw" group to an app.
system/vold/Volume.cpp @ Volume::mountVol():
Code
#
1
const char* externalStorage = getenv("EXTERNAL_STORAGE");
2
bool primaryStorage = externalStorage && !strcmp(getMountpoint(), externalStorage);
3
4
...
5
6
if (primaryStorage) {
7
// Special case the primary SD card.
8
// For this we grant write access to the SDCARD_RW group.
9
gid = AID_SDCARD_RW;
10
} else {
11
// For secondary external storage we keep things locked up.
12
gid = AID_MEDIA_RW;
13
}
This shows that whichever external storage's mount point is equal to the "EXTERNAL_STORAGE" environment variable will get the normal AID_SDCARD_RW group, and apps can get access to this storage with the "WRITE_EXTERNAL_STORAGE" permission. Any other external storage (this includes USB sticks and whatnot) will get the AID_MEDIA_RW group, and thus cannot be written to by third party apps.
The comments are somewhat confusing, as they do mention "primary SD card". The EXTERNAL_STORAGE environment variable often refers to the internal flash instead of the primary SD card. So again, maybe this is not what was intended to happen.
It is not uncommon (but also not a standard) for the internal flash to be mounted in one place and referred to as "the" external storage, with all other external storages mounted in subfolders of that mountpoint. This is a popular OEM workaround to the problem of the Android API not properly supporting multiple external storages (and Google seems to be refusing to fix this situation). This way, a third party app only needs to know "the" external storage location, and due to the subfolder structure, it can still access all of the external storages. This specific way of doing things obviously causes problems with the code posted above.
Hopefully, somebody at Google can shed some light on this situation.
Update #1
This has been discussed on Android Developers' weekly Google Plus hangout, as well as some other chats with Google engineers.
None of them had any idea what is going on here - yet. But we have gotten a promise that this specific issue will be looked into, and they will get back to me on this - I hope it will be in dialog!
I also wanted to share with you that even devices like the SGS2 (international) suffer from this issue on ICS, though Samsung has worked around it using a very ugly permission hack. The problem is still there, even if it doesn't show up.
Update #2
By request, here's the way Samsung has worked around the problem in their ICS builds for the SGS2:
/system/etc/permissions/platform.xml:
Code
#
1
<permission name="android.permission.WRITE_EXTERNAL_STORAGE" >
2
<group gid="sdcard_rw" />
3
<group gid="media_rw" />
4
</permission>
Well, this is certainly an interesting way to go about working around the problem. What is happening here is that if you ask for the "WRITE_EXTERNAL_STORAGE" permission, you also get the "WRITE_MEDIA_STORAGE" permission - at no extra charge! It should be obvious that this is not in any way a proper solution to the issue.
What makes this even more interesting is that this means at least Samsung engineers are apparently well aware of the problem.
Still looking forward to hear what Google has to say about all this. It worries me that they added code that - based on the code comments - is meant to restrict even more access, while not providing any clear reason why this should happen.
Update #3
An observant commenter on the XDA portal article stated that this issue is patched in CyanogenMod:
Code
#
1
// Originally, non-primary storage was set to MEDIA_RW group which
2
// prevented users from writing to it. We don't want that.
3
gid = AID_SDCARD_RW;
That's nice for CyanogenMod users, but doesn't help stock users. The question remains - WHY ?! There doesn't seem to be any reason for this stuff, and it's causing real-world issues to real-world devices for real-world people!
Comments
You may use tags like [b], [u], [i], [url], [quote], [code], [pre], etc

Really interesting topic i would like to know why this is happening.
;)
We are facing this problem with ICS on Galaxy S + family phones which we cant access the internal sd card after facing encryption unsuccessful.
http://forum.xda-developers.com/showthread.php?t=1447303
No real solution has come up yet.
Cyanogen himself has posted some thoughts about the subject but nothing that will really help is found.
Are you able to help us out on this?
Same issues are facing in other threads in Vibrant and Nexus S also.
Thank you in advance
The situation is terrible, SD card support have to be improved an Android. Some devices have 1 sd card, some other 2 cards, SDK supports only 1 SD card ... horror
Logo USB drives,Branded USB Flash Drives
Nice topic !!
Unbelivable, that´s like bullshit from the "Fruit Handy"
Hobe they´ll fix it soon
I wonder if another clue is not the fact that Google is not including an SDCard slot on their new Nexus tablet. Based on that clue I'd say waning us off SD store is on purpose and Google is trying to "slip it in" on the consumer.
For shame for shame for shame Google!
pleaase, tell us about what you found out in the meantime.
I didn't have the problem with galaxy tab 10.1 and galaxy sII but I have it with SIII and Tab 7.7
hope it will be fixed soon, it is unbelievable.
At least on the 7.7, the ICS update fixes the problem. Samsung is using the permission hack I described in the article again. It's a working solution, even though it's not pretty.
Anyone tell me what stock apps could be moved from internal storage on the other Samsung phones/tablets to user-mounted SD storage via the Applications Manager move-to-SD button pls.
Also, some non-stock apps which could be moved/mounted on the user-mounted SD storage on Samsung phones/tablets.
I want to show Samsung that their product (S3),at the moment, is *not-fit-for-purpose*, ie, the taking away of the ability to mount apps on user-mounted SD cards is regressive and unexpected by reasonable users of their devices.
And with navigation maps and games that have sizes around 1-2GB,11 GB is not much.
I have never seen the move-to-SD button on my S3.. :( and i have a 32gb card in it that only the camera uses..
The original Android design seems to be that all internal storage should be dedicated to apps and their related data, while removable media was to provide bulk file storage. But this changed once it became fashionable with high internal storage devices (hello iPhone), and the OEMs hacked around it via the "internal as external" move.
Then later came the "move to SD" as a response to the earlier hack reducing the app capacity, a feature that seems to be limited to whatever is designated as "primary external".
The change to media seems to be linked to the change to MTP as the transfer protocol between device and computer btw. I have seen some scattered references that apps should move to using the MTP protocol internally to interact with external storage media, but nothing definitive.
do not give up on this. at least we would like to know, WHY google changed this.
I've got 10 days left to return my 16Gb S3 to Amazon as "unfit for my intended purpose". I think over the 2 years I think I'd be wanting to keep it, 11Gb for Apps will not be enough so I'd need at least the 32Gb version
However, on a positive note here is an navigation App that supports the "external" SDcard for its offline maps ===> Osmand / Osmand+ (Settings - General Settings - Storage Directory).
And for those of you who think that 16 GB internal memory is enough - I can tell you that offline maps can take up infinite GBs of memory.
Let's hope that Samsung releases a "fix" for the topic - as I think Google have something planned for "our" futures :-(
regards and thanks
Snoopy
Google should see that this is not benefitting anyone and is seen as a huge dissatisfier.
I will keep following this topic...
i have the impression you have given up on that topic. is that true!? did google just refuse to supply an explanation for us?
No business owner/manager in their right mind will allow customer details, product data, transaction information, and who knows what other commercially related sensitive database content, into an environment such as a Cloud.
All that needs to be heavily restricted to and from removable storage for reasonable security (wifi can be disabled in a split second if necessary, and a card pulled), and preferably with the applications drawing from them too.
To me Google (and Apple) seem to be eagerly slashing their own throats (I know several people already who have abandoned ideas of buying a Nexus 7, due to the lack of such essential features, and I for one will never consider buying such a device).
The 'killer application' I need for an Android tablet, still isn't available, and if this nonsense carries on, even if it is developed, it will not be possible for those that really need it, to even consider buying it.
Google have now had at least 5 months to resolve this inexcusable situation, and that's at least 4 months too long to be waiting for the solution. If this is a demonstration of how borked in the head their approach to business and their customers is, then sorry, but my present tablet and phone will be my last Android devices.
We collected 4 [heavily discounted] new Proline Mirage 10.1 tablets that came with 3.2 Honeycomb.
We too found that we cannot set the camera to save on the Micro SD cards. Also some of the file managers, Bluetooth transfer utils and FTP clients will not recognize the Micro SD.
I found one FTP app that allowed me to edit the default path, so result.
Another strange thing is the file manager I found will not create directories[folders] or write to the Micro SD.
It's all a little frustrating after the excellent SD support we enjoyed in Gingerbread on the Galaxy Tabs!
Basically this means that the secondary sd is reserved for media and only for that. Fine example how to render a technically possible thing to unusable by some overconfident policy decision.
As I have a company-issued phone, I am forbidden to re-flash it to cyanogen...
Dear Google & Samsung: thanks for the 'feature'.
media_rw is used to lock down /data/media so ONLY /system/bin/sdcard can write
that app then provides a fuse filesystem at /sdcard, forcing all files to be owned by sdcard_rw and simulates fat filesystems
in my case, i cant make a symlink to try and abuse things, because the fuse fs doesnt allow it, and i dont have media_rw
There is a perfectly good reason for this, good for google that is. Is the same reason nexus devices don't have SD cards and chromebooks have crappy storage. They're tying to make it difficult even for OEMs that ship devices with external storage.
Google wants everyone to use their cloud services not local storage. Google music, youtube, gmail, docs, all these are in the cloud and that's where google wants you to be. That's where ads are too.
Within the 7:
<permission name="android.permission.WRITE_EXTERNAL_STORAGE" >
<group gid="sdcard_rw" />
</permission>
----- vs the 10 -------
<permission name="android.permission.WRITE_EXTERNAL_STORAGE" >
<group gid="sdcard_rw" />
<group gid="media_rw" />
</permission>
That's it. Bad news for app dev. Who wants to make a legitimate app system requirement that the device be rooted.
Thanks again.
I have a Samsung Galaxy S2 (GT-I9100) with CyanogenMod ROM 10.1, Nightly.
In /system/etc/permissions/platform.xml I have
Adding
Trying to access the external sdcard throws securityException: “destination must be on external storage” anyway.
Is there anything else to change?
Thanks.