Use SharePoint Searchservice to get content from public site

How to crawl a public website content into a SharePoint site?

Just now i developed this module, the scenario is:

I have a page in SharePoint and I want to search content from other public websites and want to show the result in a data source into my Page.

First you need to configure search service.
Create scope, each scope should be used for an individual website url, so that we can filter result on base of scope in our query.

Now once you are done with configuration and settings. Lets start our coding in Visual Studio.

First add the reference of below service into your project.


C# Code:

    protected void btnSearch_Click(object sender, EventArgs e)
    private string GetXMLString()
        //StringBuilder xmlString = new StringBuilder("<QueryPacket xmlns='urn:Microsoft.Search.Query'><Query><SupportedFormats><Format revision='1'> urn:Microsoft.Search.Response.Document:Document</Format></SupportedFormats><Range><Count>1000</Count></Range><Context><QueryText language='en-US' type='STRING'>");
        string scope = string.Empty;
        if (ddlSite.SelectedIndex == 0)
            scope = "Website1"; //Scope Name created in Search Service

            scope = "Website2";//Scope Name created in Search Service
        StringBuilder xmlString = new StringBuilder("<QueryPacket xmlns='urn:Microsoft.Search.Query' Revision='1'>" +
 "<Query><SupportedFormats><Format>urn:Microsoft.Search.Response.Document.Document</Format>" +
   "</SupportedFormats><Range><Count>1000</Count></Range><Context>" +
     "<QueryText language='en-US' type='MSSQLFT'>SELECT Title, Path, Description, Write, Rank, Size,HitHighlightedSummary FROM Scope() WHERE FREETEXT ('" + queryTextBox.Text + "') AND ((\"SCOPE\" = '" + scope + "'))</QueryText>" +

        return xmlString.ToString();

    private void SearchResult()
            // Instantiate the Web service.
                    resultsLabel.Text = string.Empty;
                    QueryWebServiceProxy.QueryService queryService = new QueryWebServiceProxy.QueryService();

                    //queryService.Credentials = System.Net.CredentialCache.DefaultCredentials;
                    //above line did not work for me, then i used below line.
                    NetworkCredential credentials = new NetworkCredential("username", "password", "domain");
                    queryService.PreAuthenticate = true;
                    queryService.Url = "http://<sharepointsite>/_vti_bin/search.asmx";
                    queryService.UseDefaultCredentials = false;
                    //queryService.useDefaultCredentialsSetExplicitly = true;
                    queryService.Credentials = credentials;                    
                    System.Data.DataSet queryResults = queryService.QueryEx(GetXMLString());                   
                    resultsGrid.DataSource = queryResults.Tables[0];
        catch (Exception ex)
            resultsLabel.Text = ex.ToString();
            resultsLabel.ForeColor = System.Drawing.Color.Red;
            resultsLabel.Text = "Search is unavailable!";

    protected void resultsGrid_PreRender(object sender, EventArgs e)
        resultsGrid.UseAccessibleHeader = false;

ASPX Code:

            <asp:TextBox ID="queryTextBox" runat="server"></asp:TextBox>
            <asp:DropDownList ID="ddlSite" runat="server">
                <asp:ListItem Value="Websiteurl1">Website1</asp:ListItem>
                <asp:ListItem Value="Websiteurl2">Website2</asp:ListItem>
            </asp:DropDownList><br />

            <br />
            <asp:Button ID="btnSearch" runat="server" Text="Search" OnClick="btnSearch_Click" Width="76px" />
        <br />
        <asp:Label ID="resultsLabel" runat="server" Text=""></asp:Label>
        <br />
        <asp:GridView ID="resultsGrid" runat="server" OnPreRender="resultsGrid_PreRender" EnableModelValidation="True" EmptyDataText="No Result found" AutoGenerateColumns="false">


                        <span style="color: blue; font-weight: bold"><a target="_blank" href='<%# Eval("Path") %>'><%# Eval("Title") %></a></span>
                        <p style="color: black;"><%# Eval("HitHighlightedSummary") %></p>
                        <span style="color: green;"><%# Eval("Path") %></span>
                        <br />
                        <hr />



  1. The given information was very excellent & Great tips, and awesome way to get exert tips from everyone,not only i like that post all peoples like that post,because of all given information was wonderful and it's very helpful for me.
    Sharepoint Training in Chennai

  2. Excellent post ! Thanks for sharing such a useful post.

    seo training center in chennai


Post a Comment

Popular posts from this blog

CAML Query in SharePoint 2010

Calling REST APi from server side code - C#

All about SharePoint 2010 Content Type Hub

SharePoint Interview Questions and Answers

Calling ASP.Net WebMethod using jQuery AJAX