Using blocks for connection pooling
To wrap up block appreciation week, we'll look at how blocks can be used for connection pooling. This is a common pattern in Ruby libraries that manage resources, such as database connections, where we want to have a limited pool of connections that can be checked out, used and then returned to the pool when complete.
An example of this pattern is seen in ConnectionPool#with from the connection_pool:
require "connection_pool"
PG_POOL = ConnectionPool.new(size: 5) { PG.connect("postgres://localhost/mydb") }
PG_POOL.with do |conn|
conn.exec("SELECT * FROM users")
end
Let's implement a simple connection pool for persistent Net::HTTP connections as a learning exercise:
require "net/http"
class HTTPPool
def initialize(size:, host:, port: 443)
@connections = Queue.new
size.times do
@connections << Net::HTTP.start(host, port, use_ssl: port == 443)
end
end
# Check out a connection, yield it to the block,
# then return it to the pool — even if the block raises.
def with
connection = @connections.pop
yield connection
ensure
@connections << connection
end
end
pool = HTTPPool.new(size: 5, host: "jsonplaceholder.typicode.com")
threads = (1..10).map do |id|
Thread.new do
pool.with do |http|
response = http.get("/posts/#{id}")
puts "[Connection #{http.object_id}] Post #{id} is #{response.body.bytesize} bytes"
end
end
end
threads.each(&:join)
# Output:
# [Connection 19976] Post 1 is 292 bytes
# [Connection 19992] Post 2 is 290 bytes
# ...
Note how we use Queue for thread safety (previously covered in reason #115), and ensure to guarantee that the connection is returned to the pool even if the block raises an exception.
This is still a very basic implementation, and there are many ways to improve it (e.g. adding error handling, closed connection handling, timeouts, etc.), but it serves well to illustrate the pattern.
Hope you enjoyed block appreciation week. Blocks really are the gift that keeps on giving!