Cloud storage services have seen a massive increase in the number of users in the last few years - both in the personal storage space and the business use case. This increase has come with a lot of scaling challenges for the service providers. One such challenge is to implement a good resource sharing management system. The users may want to share their content with others; this is fine in ordinary conditions but when a user shares the content with a large audience, your service takes a hit due to hotlinking.
There are many other problems in this space. The content that has been shared might be of illegitimate origins or might contain offensive material, so the service becomes a vector for illegal activity. This is particularly troublesome in the case of pirated content. Another problem is that the shared resource might cause an unintentional denial of service attack on the service in case it is shared widely. The service would not be able to collect any meaningful analytics either. What if the user wants to consume the content themselves but cannot login every time to reach it? This is very common when a download manager is used for fetching the resource, or when the user wants to resume the resource download at a later time. What if the user wants to share the content with a limited set of people and would like that the resource URL expires after a certain time period? Solutions to all these problems and more find their use in cloud storage services like the Amazon S3, Google Cloud and Virtual Data Rooms.
URL signing is a scalable solution to the problems mentioned above. The idea is very simple - each resource URL is generated in a way that it is unique and is identifiably linked to the creator of that URL. This is done by including an identification object in JSON format in the URL as a parameter, which is encrypted as per the JOSE standard. This object can contain a number of claims which identify the issuer of the URL, the expiration time, the start time, the scope of sharing (which identifies who all have access to the content other than the creator), and the sharing policy (public vs. private vs. login required). When a request is received, the service provider verifies it using Public-key cryptography and denies all invalid requests.
URL signing comes with some caveats too, the biggest one being difficulty in implementing caching strategies. Let’s say that a user looks at a picture on a website. This picture is served to the user via a signed URL. If the appropriate headers are set on the resource, the browser will cache it and link it to the URL it came from. However with signed URLs, unless you maintain a state on the server side, or design a stateless algorithm that issues the same signed URL within a specified duration, the browser will not be able to leverage the cached image since the URL signature would change the next time the user looks at the picture. This is also problematic when the resource has a large size, so it takes considerable time to download it. In that case the user may want to download a portion of the resource later on but resume would not work if the signed URL changes. The increased degree of difficulty in implementation is well rewarded with the benefits that come with signed URLs though.
Both Google Cloud and Amazon S3 provide first class access to URL signing in their platforms, which differ a little in their implementation details. More details can be found in their respective documentations here and here.