Remove items from a huge SharePoint list

At a certain client of mine, there was a huge list consisting over 67 million items. This is well over the suggested 30 million item limit. This list was filled with random information and was adding lots of data to the dbo.AllUserData table so we decided to remove it. After trying several approaches, we couldn’t get past the huge DELETE statement SQL builds in order to remove the list. This statement was so large, it even crashed the SQL server. One approach that worked was using the GUI to delete items. As you can guess, this would take a huge amount of mandays to click away the list 😉

Thankfuly SharePoint accepts batch updating items so I’ve written a script to delete those items in no time (subject to your processing power).

$web = get-spweb http://
$list = $web.lists | ? { $_.title -eq "<<ListTitle>>" }
$spQuery = New-Object Microsoft.SharePoint.SPQuery
$spQuery.ViewAttributes = "Scope='Recursive'";
$spQuery.RowLimit = 100
$caml = '<OrderBy Override="TRUE"><FieldRef Name="ID"/></OrderBy>' 
$spQuery.Query = $caml 

do
{
    $listItems = $list.GetItems($spQuery)
    $count = $listItems.Count
    $spQuery.ListItemCollectionPosition = $listItems.ListItemCollectionPosition
    $batch = "<?xml version=`"1.0`" encoding=`"UTF-8`"?><Batch>"
    $j = 0
    for ($j = 0; $j -lt $count; $j++)
    {
        $item = $listItems[$j]
        write-host "`rProcessing ID: $($item.ID) ($($j+1) of $($count))" -nonewline
        $batch += "<Method><SetList Scope=`"Request`">$($list.ID)</SetList><SetVar Name=`"ID`">$($item.ID)</SetVar><SetVar Name=`"Cmd`">Delete</SetVar><SetVar Name=`"owsfileref`">$($item.File.ServerRelativeUrl)</SetVar></Method>"
        if ($i -ge $count) { break }
    }
    $batch += "</Batch>"

    write-host

    write-host "Sending batch..."
    $result = $web.ProcessBatchData($batch)

    write-host "Emptying Recycle Bin..."
    $web.RecycleBin.DeleteAll()
}
while ($spQuery.ListItemCollectionPosition -ne $null)
$web.Dispose()
Advertisements

4 thoughts on “Remove items from a huge SharePoint list

  1. Pingback: Items bulk deletion | Christopher Clement's Blog

  2. Pingback: Suppression en masse d’éléments | Christopher Clement

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s