Skip to main content
SharePoint Crawled and Managed Properties

SharePoint’s `Crawled’ and `Managed Properties’ for Search

When we start learning SharePoint, we focus more on columns and site columns. However, in SharePoint 2013, the major feature that everyone talks about is the `search’ feature and what can be done with it.

To do anything with search related features, we first need to understand the difference between`Crawled’ and `Managed Properties’.

The Basics of SharePoint Search

One of the biggest issues with SharePoint, is how SMB’s  know about search and the time and effort that needs to go into it. It requires many changes starting with its configuration and moving on from there to topology, content sources and crawl schedules.

Pretend for a second that the building where you work is your SharePoint farm. It is brand new and completely empty. The rooms represent your SharePoint sites. To be able to ask the search engine what you have in each of these rooms, you will first need it to run around the entire building and take notes of everything. This is what we call crawling. How often the crawling happens will determine the search result “freshness” amongst other settings like the crawl type.

SharePoint Search

Everybody assumes that the columns created by them in their lists and libraries are now searchable. However, most often this is not the case.

SharePoint Search Crawled Properties

A crawled property is content and metadata that is extracted from an item, such as a document or a URL, during a crawl. A crawled property can be an author, title, or subject. This means that it has passed over our columns and the metadata assigned to each element. We do not have any control over the creation of “Crawled Properties”. These are the list of results of columns crawled in some way, though we are over-simplifying it.

A crawled property by itself is not useful when we try to build or run search queries or even display the value of this property in search results. Picture the crawled properties more as the information collected by that crawl that has been going around our building in our previous example. The crawl goes through our sites, lists and libraries to find our content and picks up the value in our columns and stores them as crawled properties.

SharePoint Search Managed Properties

The Managed Properties is where it all happens. If we need anything to be search related: Search Results, Refiners, Display Templates, Content Search etc., then we will need to understand how to create and manage these “Managed Properties”.

Each Managed Property is by default mapped to one or multiple “Crawled Properties”. Assume that our organization has many lists and libraries. In them, users (logged in Users) created columns like “Customer Name” and “Client”.

For the organization, the columns “Customer Name” and “Client” represent exactly the same content, but not for search. For search, they are just crawled properties and both very different ones at that, because they do not share the same name. Since they are only Crawled Properties, if someone searches for all documents where Client=Trigent then the result will be nothing at all. Because no search related feature works with crawled properties, we need to create a Managed Property called “Customer” and assign both crawled properties to it.

In some scenarios though, we may find that a Managed Property has already been created for our Crawled Property, automatically. That’s because as always, there are exceptions. If we create a Site Column and assign it to a list or library, when the indexer crawls, it will automatically create a Managed Property for it. Regardless of it being a site column or not, Managed Metadata columns will always have Managed Properties created for them.

Where do I create and manage these Properties?

Now we understand the difference between the two at a high level, we will need to know how to see the crawled properties collected and created by our search engine as well as the existing managed properties. Luckily for us, in SharePoint 2013 we no longer have to go in the Central Administration of SharePoint or Office 365 to create them. It is still what I would prefer as these will be global and centralized for all web applications associated to this search service application. It’s not unlikely for specific Site Collections to need their own Managed Properties and isolate them there.

To navigate, create or edit these properties, we simply need to find the Search Schema either in the Central Administration for the search service application or from site settings of a site or site collection.

Search Service Application

 

How to create a Managed Property for columns, which are created in the library/list:

  • Open central administration in SharePoint 2013 On-Premise environment.
  • Application Management –> Manage Service Applications (under Service Applications).

Service Application Management

  • Click Search Service Application, created in the current environment.

Search Service Application

  • Click Search Schema in the left navigation

Search Administration

  • Click New Managed Property as shown below

SharePoint Service Application

  • Provide the value for the Property Name, description, select type.
  • Check, if the managed property needs to be Searchable, Retrievable ,Queryable and if it needs to store the multiple values.

New Managed Properties

  • Click Add a mapping button.

Mapping Button

 

  • Search for the mapped property, add it to the Managed property. Mapped property will be created for all the columns with ows_ as prefix

Crawled Property Section

  • Click OK button. Now, the managed property will be created.

SharePoint can crawl data from different sources. The data in various source system have metadata which can have different name but it refers to the same information. As an example different systems can store the information about the Author in various systems with the name as Author,CreatedBy, Writer, Owner, and in case of emails it stored in the field. But all these fileds represent same information which represent who has created this.

Now SharePoint crawl these various system all these properties become Crawled properties. In SharePoint we can group all these crawled properties under one Managed property.

SharePoint Crawled and Managed Properties

The problem with optimistic title override and the title managed property

The problem we’ve been experiencing with SharePoint Search is it’s over willingness to help improve search results. We see, there is an integrated feature built in search that will extract what it thinks is the Title of our document and show that in the Search results, regardless of the value we entered in the Title column. The search engine will take, for example, our headings in a Word document or the first slide with larger text in PowerPoint and override any other title we may have specified. This was very unpleasant as we can imagine, many documents would often have the same heading as they were the result of a filled out form. As a result, SharePoint would hide our documents in search results calling them duplicates.

In SharePoint 2013, with the October 2013 CU or SP1 installed on server, we will now be able to see this special “Crawled Property” in our Title Managed Property.

SharePoint Title Managed Property

SharePoint Search Results and other search related features can only show and work with managed properties. If the Title is being displayed  as the wrong one, then we need to check and verify the Managed Property responsible for it and see what crawled properties are assigned to it and in what priority of order. Before, we could not see this special crawled property that had the highest priority and in turn could not adjust it.

Rajesh Kumar VSNK

Rajesh Kumar VSNK

Rajesh Kumar VSNK works as Module Lead with Trigent Software. With 10+ years of experience in building and designing internet/intranet based systems, Rajesh has considerable (over 7 years) experience in SharePoint Technology – SP 2013, SP 2010 and MOSS 2007. He also has experience working on migration projects from MOSS 2007 to SP 2010 and MOSS 2007 to SP2013.