A key aspect of any software user experience is the response time from submission of a request until the result is returned, usually in the form of an updated User Interface (UI). In the lab, where we develop new releases, we measure response times for a variety of transactions, such as opening a community, downloading a file, or sending an e-mail, under a variety of workloads. These are system response times. For development purposes and comparison between releases, it makes sense to eliminate all other parts of the total end user response time. But what matters to the subscriber is the end user response time, and that includes not just the system response time, but also the time needed to communicate across the network between the system and the user. For a cloud service, because that communication takes place across the internet, response times are less predictable. Depending on the nature of the protocols being used in any given transaction, and the network latency between the user and the system, that added piece of response time may vary with end user location. Furthermore, network latency is not a constant; it's not merely a function of the end user's physical and logical location on the network. It also depends on the network load at the time you measure latency. And on any content caching in place. So end user response times depend on at least the type of transaction being submitted, on the route taken across the internet, on network workloads at the time, on caching conditions, and on system workload at the time. Several of these, the cloud service provider has no control of. As a result, it is not possible to offer a map of end user response times based on location. Some cloud service providers offer a speed test tool, which will determine the latency between the user and the service, but the results can only be taken as general guidance. To offer a realistic impression of system response times in a pre-sales situation, the provider needs to offer time limited trial accounts, which the prospective subscriber can exercise in ways (timing, location, etc) that emulate the intended use. And for development purposes, we need to test the service from locations representative of various subscribers' conditions. Some of these locations can be simulated with systems that add latency to network communications. The advantage of simulated latency is the ability to control it, so we can compare successive releases, when they work under identical latencies. As discussed in Cloud Difference #19: Monitoring is Central, providers need to "see what users see". That thought surely extends to response times, but due to the nature of the internet and the protocols that route traffic across it, two identical transactions submitted in rapid succession can have different response times because they were routed along different paths to reach the data center. For that reason, end user response times are generally statistical measures, either average response times, or 90% percentile response times, or similar. We measure from select strategic points around the globe, which gives us a best case estimate for response times from those locations. But that still doesn’t include what is often referred to as the "last mile": the network segments between the subscriber and the nearest low latency segment of the internet. That last mile may consist of both low bandwidth internet segments and segments within the subscribing company's own intranet between the end user and their company's web facing proxy servers or gateway. We can simulate a generic 'last mile', but every company has different network characteristics. So we address response times at three different locations: (i) at the system itself, (ii) across the network including a set of strategically placed edge caching servers, and (iii) at end user locations within subscribing companies. For the first two locations, we can and do take action to improve response times, by optimizing our services code, and by optimizing caching design and server network. However, in the latter "last mile" scenario, we typically do not have access to the subscriber's internal network, so it's the subscriber, rather than the provider, who must take action in cases where significant latencies in the last mile affect end user response times. We have seen examples of subscribing companies' internal environments intercepting packets and interfering with our services and response times. They are all configuration issues that can be worked out, but it explains why the provider usually can not make a blanket statement as to what end user response times will be. Detailed configurations (security scanning, proxy configuration, bandwidth, etc) in the subscribing company's network affect end user response times.