Ref: http://highscalability.com/blog/2014/5/12/4-architecture-issues-when-scaling-web-applications-bottlene.html
performance of web application is used to mean several things. Most developers are primarily concerned with are response time and scalability.
Two ways of adding more hardware are
- Scaling Up (vertical scaling) :– increasing the number CPUs or adding faster CPUs on a single box.
- Scaling Out (horizontal scaling) :– increasing the number of boxes.
Scaling out is considered more important as commodity hardware is cheaper compared to cost of special configuration hardware (super computer). But increasing the number of requests that an application can handle on a single commodity hardware box is also important. An application is said to be performing well if it can handle more requests with-out degrading response time by just adding more resources.
Ref: http://venkateshcm.com/2014/05/How-To-Determine-Web-Applications-Thread-Poll-Size/
Thread Pool: In web applications thread pool size determines the number of concurrent requests that can be handled at any given time. If a web application gets more requests than thread pool size, excess requests are either queued or rejected.
Please note concurrent is not same as parallel. Concurrent requests are number of requests being processed while only few of them could be running on CPUs at any point of time. Parallel requests are number of requests being processed while all of them are running on CPUs at any point of time.
동시 요청이란 한 순간에 일부가 연산 수행되는 반면, 병렬 요청이랑 동시에 연산 수행되는 것을 의미함
In Non-blocking IO applications such as NodeJS, a single thread (process) can handles multiple requests concurrently. In multi-core CPUs boxes, parallel requests can be handled by increasing number of threads or processes.
In blocking IO applications such as Java SpringMVC, a single thread can handle only one request concurrently. To handle more than one request concurrently we have to increase the number of threads.
CPU Bound Applications
In CPU bound applications thread Pool size should be equal number of cpus on the box. Adding more threads would interrupt request processing due to thread context switching and also increases response time.
Non-blocking IO applications will be CPU bound as there are no thread wait time while requests get processed.
Little’s law applied to web applications
The average number of threads in a system (Threads = Users) is equal average web request arrival rate (WebRequests per sec = TPS), multiplied by the average response time (ResponseTime).
무조건 thread pool size를 크게 잡으면, context swiching이 너무 많이 일어나서 성능 저하가 발생한다.
