問題描述
在集群環境中讀取數據庫表的最快方法 (Fastest way to read from database table in cluster environment)
What would be a best approach to read from very big database table in clustered environment.
Lets say we need to read huge DB table as fast as we can and then send this data to jms queue. And we would like to avoid the same data to be read since it will require processing, so preferably no intersections. And this application to be deployed in jboss cluster so nodes should some how to communicate.
So in one node case ‑ non clustered environment I just can have one process reading the table. In two node case ‑ this reading should be some how coordinated to avoid the same data to be read by both nodes... Three nodes etc...
There is no knowledge on how many nodes would be in target environment, nodes can communicate using db table or jboss cache
So it is clear that read in blocks or pages per process will give maximum performance.
And it would be easy task in simple java multi threading environment since we know how many threads would be reading and it easy math how to divide in pages and assign read of page to a single thread.
But in unknown how many nodes scenario there should be some protocol between nodes to communicate and optimize reading.
‑‑‑‑‑
參考解法
方法 1:
As you have to keep huge DB data distributed I'd suggest you to take a look into some kind of distributed hash tables. I used GemFire in one of enterprise project with the same requirements and it's well‑proven. But you always have a limit of max DB connections so you can't grow limitless.
(by d3ni5、Viktor Stolbin)