MaxClauseCount Exception Lucene Net Search BooleanQuery TooManyClauses

Recently my client had a problem with the Lucene.NET search and they were getting the following Exception:


Exception: Lucene.Net.Search.BooleanQuery+TooManyClauses
Message: maxClauseCount is set to 1024
Source: Lucene.Net
at Lucene.Net.Search.BooleanQuery.Add(BooleanClause clause)
at Lucene.Net.Search.RangeQuery.Rewrite(IndexReader reader)
at Lucene.Net.Search.BooleanQuery.Rewrite(IndexReader reader)
at Lucene.Net.Search.IndexSearcher.Rewrite(Query original)
at Lucene.Net.Search.Query.Weight(Searcher searcher)
at Lucene.Net.Search.Searcher.Search(Query query, Filter filter)
at System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr fp, Object o, Object t, EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

The fix was to set maxClauseCount to something bigger.
A quick research showed that the BooleanQuery class in BooleanQuery.cs Lucene.NET has a:

private static int maxClauseCount = 1024;

and there is a Set and Get for that:

///Return the maximum number of clauses permitted, 1024 by default.
/// Attempts to add more than the permitted number of clauses cause {@link
/// TooManyClauses} to be thrown.
public static int GetMaxClauseCount()
{
return maxClauseCount;
}

///Set the maximum number of clauses permitted per BooleanQuery.
/// Default value is 1024.
///
public static void SetMaxClauseCount(int maxClauseCount)
{
 if (maxClauseCount < 1)
  throw new System.ArgumentException("maxClauseCount must be >= 1");
 BooleanQuery.maxClauseCount = maxClauseCount;
}

The solution is obvious, just raise the maxClauseCount to something you desire and in our solution we did:

BooleanQuery booleanQuery = new BooleanQuery();
BooleanQuery.SetMaxClauseCount(2048);
Advertisements

One thought on “MaxClauseCount Exception Lucene Net Search BooleanQuery TooManyClauses

  1. This is an interesting one, because obviously more clauses in your query can lead to slower performance.

    Lucene will fill in queries when your doing a RangeQuery, PrefixQuery, WildCardQuery, etc by created OR’ed clauses for each possible term. For example if you do a range query somedatefield: [20120801 TO 20120831], then Lucene will create a clause for each unique term that satisfies that query… in this case we could expect up to 31 unique clauses – if there are existing terms for each day. If you’re doing a PrefixQuery or a WildCardQuery you can see how this would quickly get out of hand.

    There are a lot of ways to approach this problem, one simple way would be in a case where you’re doing a PrefixQuery. You could create indexed fields for a shorter group of first characters. If you created an additional field for the first five characters of a field often used in a prefix query, you could choose which field to query in your code that would give you the most performance – in this case any query of five or less characters would be better served by using the five character shortened field. Likewise for date queries you could use YYYY and YYYYMM fields.

    It’s definitely something to think about. Being mindful of the amount of data you’ll need to query will inform your approach.

    Again, thanks for the nice post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s