問題描述

我正在運行一個 SQL 使用者來讀取表中的更改，這一切都很好.但是，有時會發生大量更改，然后我的查詢因內存不足錯誤而中斷，如您所料.

I'm running a SQL consumer to read changes from a table, which is all well and good. However, there are occasions where changes happen on mass, and then my query breaks with out of memory error, as you might expect.

不幸的是，我被困在 Camel 2.17.6 上，所以 SQL 組件的 StreamList 選項不可用.(雖然根據 Camel-SQL 為什么使用 StreamList 似乎加載所有 ResultSet? 由于 Spring JDBC 限制，這不能用作流列表.)

Unfortunately, I'm stuck on Camel 2.17.6, so the StreamList option for the SQL component isn't available. (Although according to Camel-SQL Why using StreamList seems to load all ResultSet? this doesn't work as a stream list due to Spring JDBC limitations.)

因此，我使用支持流列表的 JDBC 組件重新編寫了我的路由，并且一旦我增加要提取的記錄數，我仍然會出現內存不足異常.似乎出于某種原因，JDBC 組件試圖在傳遞給拆分器之前提取所有記錄.

So I've re-written my route using the JDBC component, which supports a stream list, and I'm still getting out of memory exceptions as soon as I increase the number of records to extract. It would appear that for some reason, the JDBC component is trying to extract all the records before passing to the splitter.

我現在的形式是:

from("timer:timer...")
  .to( "language:constant:resource:classpath:pathToSqlStatement/sqlStatement.sql" )
  .to( "jdbc:msSqlServerDataSource?outputType=StreamList" )
  .split( body() ).streaming()
  .setBody().simple("$body[XMLDOC]")
  .setHeader("HeaderName").xpath("xpath/to/data")
  .to("jms:topic:name");

我最初確實有一個聚合策略 UseLatestAggregationStrategy 和 split() 之后的一個額外步驟，但我已經把它去掉了，試圖刪除所有可能的導致整個查詢被保存在內存中，但我看不到我現在還能做什么.

I did originally have an aggregation strategy UseLatestAggregationStrategy and an extra step after the split() but I've stripped that out in an attempt to remove everything that could possibly result in the whole query being held in memory, but I can't see what else I can do now.

我注意到問題 camel jdbc 內存不足異常 引發了類似的問題，而且似乎沒有解決辦法.

I note the question camel jdbc out of memory exception raises a similar problem, and didn't appear to have a resolution.

(我應該注意到我遇到的內存不足錯誤確實出現在不同的地方，并且在 WinNTFileSystem 中包含了 GC 開銷限制超過，我沒有理解，還有其他與 ZippedInputStream 有關的事情，我又不明白.)

(I should note that the out of memory errors I've had do appear in different places, and included GC overhead limit exceeded at WinNTFileSystem which I don't understand, and something else to do with a ZippedInputStream, which again I don't understand.)

這是否意味著 StreamList 也不適用于 JDBC 組件，或者我必須做一些特定的事情來確保 JDBC 組件不會嘗試緩存整個結果?

Does that mean that StreamList doesn't work on the JDBC component either, or do I have to do something specific to ensure that the JDBC component doesn't try to cache the whole results?

推薦答案

StreamList 從 v.18.x 版本開始，camel-sql 支持輸出類型.在早期版本中，camel-sql 組件加載 內存中的結果集作為列表.我認為在camel-sql 2.17.x版本中無法避免.

StreamList output type is supported in camel-sql since version v.18.x. In earlier version, camel-sql component load the result set in memory as list. I don't think it can be avoided in camel-sql version 2.17.x.

camel-sql 組件將結果加載到內存中后，聚合/拆分確實適用.

Aggregation /splitting does apply after loading the result in memory by camel-sql component.

另一個相關答案.

這篇關于Camel JDBC StreamList 查詢似乎在拆分之前加載整個結果集的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！