READ THIS FIRST

Update: 3-4-2016: If you're finding this blog and looking for information related to Azure Mobile Services, I'd strongly recommend checking out Azure Mobile Apps. Azure Mobile Apps is a new version (consider it a v2) of Azure's mobile backend support. All of the same features of Azure Mobile Services are there, with a lot of other very cool features to go along. You can read more about Azure Mobile Apps, and how to transition from Azure Mobile Services, here.

Update 3-20-2015: If you are running into a cursor error when trying to upload a blob from the Android version like the following:
  Couldn't read row 0, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it

I'd recommend replacing the code which uses a cursor to get the image bytes with the following:


  Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(mImageUri));
  ByteArrayOutputStream bos = new ByteArrayOutputStream();
  bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
  byte[] bytes = bos.toByteArray();

That should fix it

Mobile Services with AndroidSince we launched Mobile Services, one of the common questions we get is how to connect Mobile Services with Windows Azure Table and Blob Storage.  Mobile Services is built off of SQL Database which is really great for many reasons.  The motivations for connecting to Table and Blob storage usually have to do with scalability, cost, and file storage.  Table storage is a NoSQL data storage implementation that offers really great scalability in addition to geo-replication.  Additionally, table storage is a good deal cheaper than SQL database.  Blob storage is specifically suited to storing files be they images, videos, or anything else.  SQL Storage makes Mobile Services a bit more straightforward but does come at the costs I just mentioned.  Today, I’m happy to announce a sample which will demonstrate how you can connect your Mobile Services into Table and Blob Storage.  It’s not quite as easy as flipping a switch and saying “don’t use SQL, use Tables and Blobs” but this should give you a pretty thorough understanding of how to take advantage of Tables and Blobs.  Additionally, after this you should understand enough to be able to use them in conjunction with the built in SQL storage if you want.  This article will cover how to handle things on the server (Mobile Services) side and the next two will cover connecting to them with the iOS and Android clients.  Source code, including the server scripts, is available for iOS and Android.

An important note on the difference between SQL Database and Table Storage

A very important thing to point out is that you can’t just replace SQL Database with Table storage and think you’ll get all of the same features. Specific design and functionality has to be taken into account when using either type of storage.  One very specific example of this has to do with querying your data.  With Table Storage, you are able to do look ups on a table against it’s Partition Key and Row Key.  That’s in.  With SQL Database you can index and look up by any column in any table.  It is extremely outside the scope of this article to discuss the advantages and disadvantages of the different storage systems and it would be impossible to provide a “one size fits all” answer to what you should use.  When in doubt, do the proper research to understand your options. 

Setting up your Mobile Service and Storage Account

The first step we need to take is to create a new Mobile Service and Storage Account.  Before you can do that, you’ll need a Windows Azure account.  If you don’t have one yet, you can sign up for a free 90 day trial here.  Next, as opposed to walking through the steps to setup a new Mobile Service yet again, you can reference the Setting up the app portion of this article.  Now that that’s done, let’s set up our Storage account.  Click the + NEW button at the bottom left of the portal and choose DATA SERVICES and then STORAGE:

New Storage Account

Next choose QUICK CREATE, name your account, and choose it’s location:

Storage Account Name

Click the CREATE STORAGE ACCOUNT button and your account will start being created.  Once it is, click into your storage account.  Before we can move back into our Mobile Service, we need to get our storage account key.  Once in your storage account, click the MANAGE KEYS button at the bottom:

Manage Storage Keys

In the window that pops up you’ll see a primary and secondary key.  You’ll need to copy the name of your storage account and the primary key for use later:

Storage account key and name

Accessing Tables

Now that we’ve set up our Mobile Service, let’s take a look at accessing Tables.  Go into your Mobile Service in the portal and go to the DATA tab (it’s the link near the top under your Mobile Service name).  If you started with a new service, you won’t see any tables so go ahead and click the + CREATE button at the bottom.  We’ll name this table Tables and leave the permissions to the default with the exception of READ PERMISSION.  Set that to Everyone.  We’re only going to do that so that we can also check our tables from the browser later on. 

New tables table

After that, click the checkmark to create your table.  Once your table is generated, click on it’s name to be taken into the table.  You’ll start by seeing that your table has no records.  That’s fine and in fact, since this view is tied to what’s in the SQL Database and we’re going to use Table Storage, we’ll never see our data here.  Click on the SCRIPT tab at the top.  We’ll first handle the Insert script by replacing the default with this:

Make sure you replace the accountname and accountkey with the values you grabbed above for your storage account.  You’ll need to do that in all of the scripts going forward.  The first thing the script does is get access to the azure module.  This is a Node.js module that gives us access to a lot of different things including Table and Blob storage.  After we set our account name, key, and host variables, we call createTableService with them.  This gives us a service we can use to access Table Storage.  Next we call the service’s createTable method and pass in the tableName property that was sent into the method as JSON (into the item object).  If the table was created we respond with a 200 and the item object.  If not, we respond with a 500.  As we continue on, you’ll see that most of the scripts look very similar.  We get the account name, key, and host and use them to generate a service using the azure module.  Then we do whatever we want with that service.  Save that script and then use the drop down at the top to switch to the Delete script:

The delete script is nearly identical to the create script.  Instead of calling createTable we’re calling the deleteTable method.  Also, since the delete script doesn’t take in a JSON item like the other scripts, we need to get the tableName from the request.parameters object.  Lastly, we’re not returning any data on success, just the 200.  Save that and then switch to the Read script:

The read script uses the queryTables method and returns either the error in case of an error or the actual tables data on success.  That’s all we have to implement in order to insert, delete, and read tables.  Next we’ll look at accessing the data inside of those tables!

Accessing Table Rows

Return back to the DATA tab (you can click the big arrow pointing left if you’re still on the read scripts for the tables table).  Go ahead and create another table with the same permissions and name it TableRows.  Click on your table after it’s created and go to the SCRIPTS tab.  We’ll start with the Insert script:

Everything is the same (again) up until we get to the insertEntity method.  This method takes in the name of the table, which we’re getting from request.parameters, and the data for the table which we’ve passed over in the JSON item.  Save that and move on to the Update script:

This script is completely the same except we’re calling updateEntity instead of insert.  Let’s go on to Delete where things will be slightly more interesting:

Here, we need to send over the table name as well as the Partition Key and Row Key.  All of these values are retrieved from request.parameters because, again, we can’t pass this data though on a JSON item as the delete method only takes in the id of the item to delete.  Let’s wrap things up with the Read script:

Here we’re actually creating a azure.TableQuery to query Table Storage.  We’re doing a very simple query here and just saying “select all the rows from the table with this name”.  However, if you wanted to do more advanced querying (such as pulling all the items from a table with a certain Partition Key) this is where you would do it.  Once we’ve created that TableQuery we pass that into the queryEntities method.  On success we’ll return the rows of data that are returned from our query.

That’s all there is to the server side of accessing Table Storage.  We’ll next look at dealing with Blob Storage.

Accessing Containers

As the first part of dealing with Blob Storage, we need to see how to look at the Containers.  Containers “contain” blobs so before we can load up blobs we need to deal with what they’re in.  Return to the DATA tab for your Mobile Service and create a new table with the same permissions as before and name it BlobContainers.  After your table is created, click into it and go to the SCRIPT tab.  We’ll start with the Insert script:

Things for setting up are just a little different here than in the table scripts.  First, the host is set to the accountname + .blob.core.windows.net.  If the host name was left to tables.core.windows.net we wouldn’t be able to access Blob Storage.  Next, we’re calling azure.createBlobService in order to get a blobService for accessing Blob Storage.  That’s going to be the same for the rest of the scripts that deal with blobs.  Next, we’re checking the isPublic flag in request.parameters.  A container can be public or private (it’s a little more complex but that will do for this article).  If it’s public, I don’t need any special permissions to access the blobs inside of it.  If it’s private, it requires a bit more work but we’ll talk about that when we get down to blobs below.  Either way, we’re calling the createContainerIfNotExists method and passing in the container name from the JSON item.  Since the default is private, if the isPublic flag is set to true, we pass in some options that specify that the publicAccessLevel is for blobs.  Once created we return to the calling app.  Let’s move on to the Delete script:

Deleting is very similar to the table deletion, we just call deleteContainer and pass the container name from request.parameters.  Let’s wrap up containers with the Read script:

Here we just call the listContainers method and return the results to the calling app.  Containers are now covered.  We’re ready to talk about Blobs.

Accessing Blobs

The last step before we move to our client is how to load blobs.  Go back to the DATA tab in your Mobile Service and create a new table with the same permissions named BlobBlobs.  Once it’s created go in and let’s tackle the Insert script:

This script is by far the most complex one we’ll look at today.  Before we talk about the code, let’s talk about how we handle inserting a blob.  To insert a blob, we need to have a URL we can write to.  In order for Blob Storage to be secure, we can’t just write to http://myaccount.blob.core.windows.net/mycontainer/myNewBlobName.  Blob Storage needs to know that there is a blob there and approve us writing to it.  To facilitate that, we generate a Shared Access Signature (SAS) which gives us write (and read) access to a specific blob’s URL.  We can then return that SAS URL to the client who can read or write to the blob.  We’re going to use the Insert method to both generate new blobs and get access to existing blobs.  We could split this out and have one method for reading and one for writing, but for simplicity it’s all in one here.

So in the script after we call createBlobService we generate a sharedAccessPolicy.  Here we’re saying we want to be able to read and write and that the SAS URL should expire in 5 minutes.  If we only wanted read access, we could just change the permissions.  After that we get the sasUrl object with the generateSharedAccessSignature method.  That method takes in the access policy as well as the blob and container names.  After that, we need to build our SAS URL from the properties in the sasUrl object.  To do this, we use the stringify method of the querystring module to build our query string parameters.  Finally we return that to the calling app.  Let’s move on to the Delete script:

Back to simplicity with this as we just call deleteBlob and pass in the container and blob name.  Now let’s look at the Read script:

Here we call the listBlobs method and pass in the container name.  This will return the details for all of the blobs in our container.  That’s all for the server side scripts for accessing Table and Blob storage.

A word about how things are set up

These scripts and this sample were built to be a demonstration of how you might access Windows Azure Storage from Mobile Services.  To that end, some of the things I’ve done could certainly be done in other ways that might fit better with what you’re trying to accomplish.  For example, I used four “dummy” tables for Tables, TableRows, Containers, and Blobs.  It might make more sense for you to use one table in Mobile Services to match up to each table in Table Storage (at least for ease of knowing where things are at).  Alternatively, you could store some data in SQL Database and some in Table Storage.  As I pointed out above, instead of using one script to get read AND write access to a blob, I could split these into two separate functions / table methods.  Additionally, we could have used fields on the JSON item to specify some things (like the Table name on the TableRow update script) and then just deleted the TableName from the item before passing it over to storage if we wanted.  There isn’t necessarily one way to do any of these things. 

Finally, for more information about the azure module such as how to do more advanced queries against Table Storage, the different method overrides, as well as how to access other things like Service Bus or Queues, check out the module details in GitHub.

The Client

Now that we’ve covered everything on the server side, we’re ready to tackle the clients.  I’ve split things up so there is one article that will cover everything for Android and one for iOS.  It’s important to remember though that we’re using the same Mobile Service for BOTH clients.  There is no need to create a different Mobile Service for each client.  You can find the specific articles for each client here:

Conclusion

Today we looked at how you can easily make use of Table and Blob Storage from your Mobile Services.  You should be capable of understanding how to take advantage of all the different kinds of storage from Mobile Services now.  Remember that there are advantages and disadvantages to all of the storage providers.  Additionally, there are many different ways of doing the same things.  This sample is meant to give you the basics that you can work with to craft your own solution for your apps. 


Chris Risner


45 Comments

Guest

Hi Chris, I am creating a sasUrl and post images. The sasUrl creation is always successful which I am very happy with, however, Chinese users always fail to post images to that Url while U.S. users always succeed. Do you know what maybe going on?

Chris

Off the top of my head, I'm not sure what may be going on. I'm not sure where you're located but is it possible for you to test from China? I would advise adding logging to your application so you can find out what error they're getting when those users try to post the images.

Guest

Hi Chris, Thank you, this is just what I was looking for. I have a question: I'm planning to make an app for Android where users authenticate first and when they do so they can store and access their own files "blobs" (something like dropbox or skydrive). I think what I have to change is the insert and read scripts for the blob containers but I have no clue where to add such parameters.
Can you tell me how? I'll be very grateful!

Chris

So you'd want to use the createContainerIfNotExists method to create a container (probably per user) and then insert blobs to their specific container by generating SaaS as illustrated above.

Dave

Hi Chris, in .net, on my ReadAsync - what do I use to get the list of blobs in the container back ? I'm trying System.Collections.Generic.IEnumerable(Of String)

but no joy ?
Thanks

Chris

Hi Dave, the script above returns more details than just the names of the blobs. Possibly the best thing to do would be to add a console.log(blobs) statement before you call request.respond so you can see the format of the data. You can then create a strongly typed data object and have the readAsync return a collection of those. Hope that helps.

Craig

PUT-ting image blobs up with your example code appeared successful (NSURLConnectionDelegate reported all bytes as sent, and I got the Finished Loading call), but the container has no blobs in the Azure portal. Do you have any idea what could be happening? Thanks.

Chris

It should be a POST when you're sending the data up to blob storage. So there's that and then I'd make sure you're definitely sending up data. Nothing will fail if you try to send up an empty NSData object.

VeTaL YaVeTaL

Hi,

Just wanted to notify you that

// Go into your Mobile Service in the portal and go to the DATA tab (it’s the link near the top under your Mobile Service name).

Seems to be out-of-dated. They dont have Data tab there and i still cant find, how to edit Insert script.

Chris

It sounds like when you created your Mobile Service in the Azure portal, you choose a .NET backend vs a JavaScript one. The .NET version doesn't have the data tab. You'll need to generate a JavaScript backed Mobile Service to duplicate this post.

George Lovitz

One question I have is: Since when we create tables you state: "That’s fine and in fact, since this view is tied to what’s in the SQL Database and we’re going to use Table Storage, we’ll never see our data here." How do we view that rows/entries in the table? I went into the SQL DB that was created via the Mobile service and it is empty yet when I run the app (iOS), it shows the tables created.

Chris

If you use Table Storage, there isn't (currently) a way to view this data inside of the Azure portal. You'd want to check out the Azure Storage Explorer (there are other tools as well as Visual Studio which will let you navigate through Table Storage). Essentially in this case, you're really not using SQL at all. It just so happens that when you generate the tables for the Mobile Service, it also generates SQL Tables. However, in the scripts, you're never sending anything to SQL but instead using Table Storage. One thing to note is that this post came out before Custom APIs were available. If I was going to redo this today, I would use Custom APIs to read / write / etc to Table Storage so it didn't have the confusing SQL connotation.

George Lovitz

One other question: Is it possible to create other columns in the Blob storage? Meaning, right now when I view a container I created, it has NAME, URL, LAST MODIFIED and SIZE for columns. I would like to actually have a few other fields in there. Ideally, I would like to store two images, original and a thumbnail. Possible? Can user defined fields be created in a container just like in a table? Sorry for all questions, nube and trying to wrap my head around this. Thx in advance.

Chris

Hi George, Blob Storage is fairly simple, however, you can apply metadata to a blob allowing you to store certain extra information. Check out this link for more info: http://msdn.microsoft.com/e.... Storing an original and a thumbnail is totally possible though you'd either need to handle uploading the two different sizes from the client or you'd need to have a Worker Role or Web Job running which frequently fetches new images, downsizes them, and adds a thumbnail to blob storage. Either way you may want to consider also using Table Storage to keep track of that info.

Chris

Once you have the SAS Url you need to do a POST request as opposed to PUT. You'll have to do some searching for how to access file contents on the file system from PhoneGap's javascript.

Chris

Replace the code using the cursor to get the image data with the following:

Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(mImageUri));
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
byte[] bytes = bos.toByteArray();

mehvish

Hey Chris!
Brilliant tutorial! But when i select an image I'm getting "couldn't read row 0 col -1 from CursorWindow which has 1 row and 6 columns. Make sure the cursor is initialiazed correctly before accessing data from it."
Could you please tell me where i went wrong?

Peter

I'm also facing the same error message on android.

The error happened during getLoadedBlob, seems like blob was never created successfully. I cannot see any data from my mobileservice -> data section as well. Any advice?

} else if (intentAction.equals("blob.created")) {

//If a blob has been created, upload the image

JsonObject blob = mStorageService.getLoadedBlob();

String sasUrl = blob.getAsJsonPrimitive("sasUrl").toString();

(new ImageUploaderTask(sasUrl)).execute();

}

Peter

sorry rephrase, error happen during code section below
Cursor cursor = getContentResolver().query(mImageUri, null,null, null, null);
cursor.moveToFirst();
int index = cursor.getColumnIndex(MediaStore.Images.ImageColumns.DATA);

Chris

Replace your cursor code with this:

Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(mImageUri));
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
byte[] bytes = bos.toByteArray();

Keith

I'm thoroughly disappointed with Microsoft on how hard it is to use Azure from iOS. I am a long time .Net developer just getting into iOS, so it seemed like Azure was my best bet for cloud storage. I have simple needs for this project - just a small table with basic insert and update and query abilities. So Azure Table Storage seems perfect. I have a strong preference to using a RESTful interface to keep my code simple and avoid using unnecessary SDKs. However, after a week of trying it appears it's basically impossible to correctly build up the Signature String required by the REST interface for Table Storage - this is incredibly disappointing - a REST interface should be easy to call.

So my alternative is to use the SDK, but it turns out I can't call table storage directly from the SDK either - wow. This means I would have to write a service layer, create a mobile service in Azure, pull in the SDK and then talk to that. No thanks, I'll switch to Amazon and use their SDK to talk directly to DynamoDB.

I only leave this comment as the best way I can figure to give the feedback to Microsoft that their tooling to use Azure from iOS is greatly disappointing.

Chris

We do need an SDK for talking directly to storage from iOS, however, talking to any storage system (including Azure Table Storage, and DynamoDB) directly from your mobile app is always risky, especially if you intend to release the app to the public. If you don't intend to release an app publicly, than I can't imagine you'd be making more requests than you'd easily be able to do in the free tier of Mobile Services. Then you'd have a proxy which would protect you further from man-in-the-middle attacks as well. The best way to leave feedback that will reach everyone at Azure is to fill something out at http://feedback.azure.com.

Joel Milne

Thanks for the tutorial and sample app. Quite helpful.

I must say, that it is disappointing that in the 2 years since WAToolkit was deprecated the functionality it contained for native iOS access to storage has not been ported to the iOS Mobile Services SDK. This node.js workaround still feels a bit like a hack and I was hoping the native iOS functionality would be brought back before I needed to replace the WAToolkit.

Anyway, I thought I would suggest adding a note to this post that it's no longer necessary to create the dummy tables to access server scripts and this should be done via Custom API endpoints going forward (which is a bit less of a hack :).

Chris

Great points Joel. I'm pushing on the team to build out an iOS SDK for direct access to storage, though again with the caveat that direct access is risky unless used in the right circumstances. I'd love for you to contact me (via email or the contact form) for more information on the scenario you're looking at.

Good point on switching over to using Custom APIs. I'll add a note here soon!

Aniket Rawat

Hi Chris,
There is a limitation for number of records returned by a storage table in a single query to 1000.
How should we handle records more than 1000, either by mobile client or on table script.
Are there any example code on handling it on the iOS.

Edward

Hi Chris, I have a table which has a column for images. everything works fine until i try to send an image to this table along with other data. I get an error saying "Request body maximum size limit was exceeded.". I understand that I have to use blob storage for images. Can u please tell me how to send the image to the blob storage while sending the other data to the table and then get that image back along with the other data when I want to retrieve it?

Chris

My recommendation would be to do the following:
1. Upload your table data to an endpoint (like we're doing above).
2. The script for that endpoint should first get a Blob SAS URL (as explained below) and add that URL to the table data (so the data inserted into table storage will contain the URL for the image).
3. The script endpoint should then handback to the client the SAS URL (again as above)
4. From the client when you receive that data, you can then post the image to the SAS URL.
5. When you then query to retrieve a row, you'll hand the data back to the client which will include the URL for the blob (if you want to protect the data, you'll have to get a SAS URL each time here)
6. The client will then load the images via the URL as needed (I wouldn't recommend trying to pull the image data from the Mobile Services endpoint and return it from there)

Guest

Hi!!! Congratulations for your very helpful tutorials!!

I just have a problem!

When I want to upload an image (by creating a new Blob in the BlobsActivity screen) with an android device which contains Android 4.1.2 I can do it successfully, but when I want to upload an image from an android device which contains android 4.4.2. or higher, it doesn't happen.

I have tested this with four android devices:

Two Android 4.1.2 devices (it works without problems) and one Android 4.4.2 device and the other with Android 4.4.4 (both of them couldn't upload the image).

Then I went to the Target SDK and I updated to 19 version, also I added permission to read external storage but the error continues.

I have readed some comments with the same error from the mehvish and Peter users, so I decide to attach two images to show you what is happening in the LogCat of Eclipse, one image shows an error that Eclipse throws when the app can't upload images and the second one shows when the app can upload images succesfully.

I hope you can help us!!

So many thanks for your time!! :)

Rizo Xuxochrist

Hi!!! Congratulations for your very helpful tutorials!!

I just have a problem!

When I want to upload an image (by creating a new Blob in the BlobsActivity screen) with an android device which contains Android 4.1.2 I can do it successfully, but when I want to upload an image from an android device which contains android 4.4.2. or higher, it doesn't happen.

I have tested this with four android devices:

Two Android 4.1.2 devices (it works without problems) and one Android 4.4.2 device and the other with Android 4.4.4 (both of them couldn't upload the image).

Then I went to the Target SDK and I updated to 19 version, also I added permission to read external storage but the error continues.

I have readed some comments with the same error from the mehvish and Peter users, so I decide to attach two images to show you what is happening in the LogCat of Eclipse, one image shows an error that Eclipse throws when the app can't upload images and the second one shows when the app can upload images succesfully.

I hope you can help us!!

So many thanks for your time!! :)

Chris

Try replacing the code which creates a cursor to pull out the image data with this:

Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(mImageUri));
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
byte[] bytes = bos.toByteArray();

immi

hi Chris ! i am new to android with azure . i have created an app in android studio and a database created in window sql server ,now i imported the database to the window azure as like as your previous tutorial .now i want to connect the app with the database and do some operation on it ..what is the best tutorial for me onward .thanks

Chris

Hello. If you're doing new app and database creation, I would recommend you go the easy route of using Azure Mobile Services. That will get you set up with a service backend as well as a database to store data. The client SDK makes connecting super easy and even supports offline data sync. You can get started here: https://azure.microsoft.com...

Chris

By default, Azure Mobile Services (and now Azure App Service Mobile Apps) are built to work very well with Azure SQL Database. Absolutely nothing stops you from using Azure Table Storage, it just requires some additional work from you. Read over this so you'll better understand how to use Table Storage from Mobile Apps: https://azure.microsoft.com...

Deepak Rao

Thank you for quick response.

I went through the article but it's too complex for new bee like me. Is it possible for you to come up with step by step article in future days on this topic? I am sure its most common scenario. In my case I am storing device temperature (IoT) in azure table storage through event hub. Now I want to pull data/push data from mobile clients like android/iOS.

Leave a Comment