A utility that goes beyond the Azure CLI to bring you useful tasks against table and blob storage.
Title: Introducing the azs-util Azure Storage Account Utility Published: 10/30/2020 Updated: 10/30/2020 Keywords: Azure Storage, Azure Table Storage, Azure Blob Storage Description: A utility that goes beyond the Azure CLI to bring you useful tasks against table and blob storage. CodeProject: true Image: https://res.cloudinary.com/endpoint-systems/image/upload/v1604066948/storage_lgbfw7.png Categories: - Cloud - Azure - Storage - CLI
Azure Storage remains one of the most frequently used Azure products out there. It's a fantastic catch-all for file storage, and the table storage product is one of the cheapest, high-performance key-value storage products on the market today.
Tooling for these products, however, could use a little help. The az CLI from Microsoft does a great job of helping you administer your account at the artifact level, but it doesn't get much further deep than that. So we decided (out of necessity for an existing client) to build a CLI tool that goes into the storage account itself to give us some helpful administrative tasks.
Getting Started
The first thing to do when getting started is to set the AZURE_STORAGE_CONNECTION_STRING environment variable. This value is consistent with the environment variable for storage commands in the az cli so you can use the two tools together.
Table commands
Rename Partition Key
The table command only has one subcommand, and that is rename-partition-key or rename-pk for short. As you can expect, this is for renaming partition keys. The code for this command runs ExecuteQuerySegmentedAsync and you can set an optional batch size (default is 100) for the command to execute on. The 'old' entity is written to a new DynamicTableEntity with the new PartitionKey and properties written into it, and at the end of the batch bulk insert and delete operations are performed.
Container Commands
List Containers
The list command simply does that - list all of the containers found within a blob storage account. This will come in handy if you have a lot of containers - it gives you a nice, simple list, and a count at the bottom. Yes, you can tool together something using az and setting up output formatting, but it seems a little harder than it needs to be just to spit out container names. JMESPath might be popular at Microsoft because God forbid someone uses XML/XPath for anything anymore, but that's another argument for another time. What else does this baby do!
Delete Containers
Our delete command comes with an argument that az does not do - bulk deletion of all containers in a storage account. Simply use -a and watch all of your containers quickly disappear!
Container Size
It's always nice to get an idea as to how many blob items you have in your storage accounts, and how much space they're using. So we wrote code to go through every container, count every object, get the size for every container, and provide a nice, neat summary at the end. Our output looks like this:
cx1447: 1696 objects totaling 1.163 GB
cx1476: 16 objects totaling 0.010 GB
cx777: 32 objects totaling 0.313 GB
cx789: 112 objects totaling 0.470 GB
75 containers
0 append objects
0 page objects
8192 block objects
8192 total blobs
57220939214 bytes (57.220939214000005 GB or 0.057220939214000004 TB)
operation completed in 30.8986122 seconds.
Beautiful, innit? A nice, pretty report on the information you need.
Feel free to take a look at the source code at https://github.com/endpointsystems/azs-utils and let us know if you think there's something missing.