For the sake of discussion, I'll name and define these two tiers this way:
- SOA Service Provider (SOA-SP) -- This is the tier whose code implements the services. The code has a service API which declares the services and provides the means for clients to invoke those services.
- SOA Service Coordinator (SOA-SC) -- This is the tier whose code provides user functionality which is implemented using services in one or more SOA-SPs. It probably has a UI/GUI so the user can interact with the SOA-SC as a traditional application.
If the SOA-SC has a GUI, when the user tells the GUI to perform a task, the SOA-SC runs the task synchronously (while the user waits). It implements the task using one or more services, probably run sequentially and perhaps conditionally, but concurrent services are also possible. When a task needs to run for longer than a user wants to wait synchronously, the GUI can provide the user the means to kick off the task asynchronously. The GUI would implement the asynchronous task using a business process/workflow (such as one implemented in WebSphere Process Choreographer).
An SOA-SC and all of the SOA-SPs it uses could all run in the same process (i.e. Java virtual machine), which would be more efficient and provide better performance, but less clustering and fault-tolerance, and isn't really the idea of the whole SOA thing. Rather, the idea is that the SOA-SC runs in a process and the services it uses run in one or more SOA-SCs, where each SOA-SC runs in its own process. In theory, a single process might contain code for both an SOA-SP providing services and an SOA-SC using services and providing end-user functionality, but in practice I think an SOA-SC and SOA-SP will run in different processes with different QoS requirements. (For example, a service in an SOA-SP needs to always be available and always scale, whereas an SOA-SC that's unavailable may be annoying but permissible.)
While an SOA-SC may have the ability to directly invoke the services in the SOA-SPs it uses, a better plan is for an SOA-SC to connect to its SOA-SP through an Enterprise Service Bus (ESB). The SOA-SC gains the advantages of the ESB of not needing to know what the SOA-SPs are, how to connect to them, etc. (See ESB vs. Message Bus.)
So when you think about an SOA, don't think about it as a single application running in a single process. (This is possible, but not really the whole idea of SOA.) Although it all may be running on the server, its two distinct tiers probably running in two processes. Is that still one application? Users will think of the SOA-SC as the application, but developers will put the heavy-lifting horsepower mostly in the SOA-SP, so they'll seem like separate applications.