December 19, 2017

How to resolve the 5000 item limit on a list or document library

For more about the SharePoint solutions we offer to help boost teamwork efficiency throughout your company, visit our SharePoint Consulting page.

If you have worked with large lists in SharePoint, you have probably received errors due to the threshold limit. SharePoint is configured with a default threshold limit of 5000 items in a list or library. This means that if the items in a list or library exceed that number, any views, grouping and queries will fail.

For SharePoint on-premises versions you have the option of increasing this limit at the Web Application level in Central Administration; however, you should be aware of the risks of changing this limit in your environment. If you work in SharePoint Online (Office 365) you don’t have this luxury so you need to find a workaround to the limitation.

One of the alternatives for SharePoint Online is to index the columns that you are going to query very frequently and index them before you reach the threshold limit. The challenge with this approach is that you need to know ahead of time which columns you are going to use, and once you reach the limit you can no longer index columns.

Recently, Microsoft has introduced a new feature called Predictive Indexing that will allow you to create indices on libraries up to 20,000 items and display them in the modern UX experience. However, this feature has not yet been rolled out to all tenants.

To avoid this issue with views, one approach is to create a custom search page using features listed down below:

Content Search Web Part
Search Box Web Part
Refinement Web Part
Display Templates
Managed Properties

Custom Search Page example:

This Custom search page provides an example with all of these SharePoint features working together to enable end-users to find their information quickly using queries and filters.

Content Search Web Part

The Content Search Web Part is a web part that can display dynamic content that was previously crawled and added to the search index. Each instance of the web part is associated with a search query and shows the results for that particular query. When users browse to a page that contains a Content Search web part, a search query is automatically executed, and the corresponding search results are returned from the search index.

The Content Search Web Part uses display templates to render the results.

Display templates are HTML and JavaScript rather than XSL.
Display templates are configured for the Site and Site Collection rather than for a web part.
Display templates are applied with rules and logic.
Display templates are applied to individual result items, not the entire result set.
Display templates are used for results of all kinds, search results, content by search results, and refiners.

You can build your query and apply refiners, sorting and perform testing. This query will retrieve all items from a document library.

Content Search web part and Query Builder:

Control Display Template

This is the part of your design that does not get repeated as each item gets loaded in the Search Results. It is the container around and where you refer to any custom CSS or JS files you may want to use.

More simply, it is the box where search results will be loaded.

Display template elements:

Item Display Template

An Item Display Template is where you design how each item will look, which managed properties from the result item will be used, and where on the design. From the custom search page example above, the image below shows what is included in the Item Display Template.

Content search web part and custom item display template:

Search Box Web Part

The Search Box Web Part displays a text box where users can enter keywords or phrases to search for information. You can use the Search Box Web Part tool pane to specify properties for this web part.

Using the web part settings, you are able to change the following:

Change where the search results should be displayed – for example, show results in a custom Search Results Web Part or a custom search results page.
Query suggestions, including personal favorite results.
Links to use for a search preferences page and an advanced search page.
The display template to use.

Search Box web part and filtering search results:

Managed Properties

If you need anything to be search related such as Search Results, Content Search, Refiners, Display Templates, etc., then you will need to create “Managed Properties.”

Managed Properties are sometimes automatically created by SharePoint based on the crawl properties, mapping a specific crawled property to the Managed Property. For example, a Managed Property may be created with a name of “OWSMyColumn” and have a mapping to the “ows_MyColumn” Crawled Property.

For more information about Managed properties please see the link on the references section.

Refinement Web Part

You can filter your search results by searching only certain properties associated with a file you’re looking for. This is especially useful when looking for certain file types. Once the search results page appears, refiners become the most powerful way to improve the search results. Refiners allow you to filter your results by excluding extraneous file types, date ranges, and authors (by default).

Refinement configuration:

Custom search page with a selected filter:

Summary

Applying these available search components, including out-of-the-box configuration and minimal custom code, will enable end-users to find information faster and more accurately as well as address the 5000 item list threshold limit. Additionally, using CSS and JavaScript, you can provide a more robust solution and provide your users with a better interface and experience.