How do you manage the size of your SharePoint sites?

How do you manage the size of your SharePoint sites?

Let’s start this article by putting into context the reasoning behind site quotas, and of course, this means a description of exactly why the feature is required.

Site Quotas, in a nutshell, allows you to define a limit on the storage capacity of a SharePoint site collection. A TechNet article describes a Quota as this:

The storage limit applies to the site collection as a whole. In other words, the storage limit applies to the total size of the content for the top-level site and for all sub-sites within the site collection. If versioning is enabled, the versions in a site and the content in the Recycle Bins count toward storage limits.

Without this feature, you would have SharePoint sites blossoming into an un-controlled; file ‘trough’ areas where content could be uploaded. So, I hear people cry, “Why is that bad? Shouldn’t we just allow people to store anything and not subject them to a storage limit”?

Storage costs money. And there are different types of storage. Some companies charge departments for storage allocation. For example, if you wanted to create your own website, you’d have to pay based on the capacity of that site as one of the key drivers. Also, you, and site owners would want to know how their site is growing, how much content is there and how soon they would hit a threshold before purchasing more site space (and as such, upping the quota)…

Managing SharePoint Site Size.

When quotas are defined, SharePoint provides out of the box reports which provides a list of all content in a SharePoint site, its size, the number of items, when last modified and the location of them. These are displayed for a Site Collection Administrator. Now, I’ve had many conversations on the nature of this report and whether the site owner, or designated people in the company should be able to see this type of report on demand. That is where there may be an issue in terms of managing the site size. Because the report is available to a site collection administrator only, that assumes that the person looking at the report must be a SharePoint lead admin with knowledge of the site collection features in a Site. Some people would say “So what? I’ll make anyone who wants to view this report a site collection administrator”. Looks of horror from the SharePoint community as someone hails, “Hey, 30% of my companies business end users are site collection admins in SharePoint”.

So what do we do instead in dealing with site size reporting? Here’s some options found on my travels; is your scenario is below? :

  • Print out or screenshot the quota screen and pushes it on to the relevant site owners. This may work in a small organization in say a couple of hundred sites. But what about sites where they are in the thousands or tens of thousands with site quotas set?
  • Use a third party application to automate output the Site Quota via XML and push it into Excel then put that as an Excel HTML page on a site. The owner looks at the size and requests an increase on demand.
  • Use a third party tool to monitor the site size and automatically set it to the next quota level and send out an email to the site owner and administrator. Ok, sounds cool? Maybe not I hear people say! I don’t sense anyone managing that site size and growth rate?

Am sure there are others, and probably more creative and more outlandish than the ones that I have listed above. I suspect that because of this, there is no ‘best practice’ out there concerning governance on site using site quotas. Instead, companies using SharePoint have gone down a road of choosing their own ways of managing site growth and size.

Here’s my stab at defining best practice for SharePoint Site Storage Governance.

  1. Identify the site in terms of its functionality, the kind of content that it will hold, what features and products may connect into it, the number of potential users.
  2. Related to the above. Understand the current organization model for storage requests. Are there costs associated and do these relate to SharePoint. If yes, then map the quota templates to the cost charges and ramp them up accordingly. If not, still apply quota templates at least to keep an eye on site growth and help site owners to manage site growth and monitoring.
  3. Find out who the owners are who need to know about site growth. No, you are not going to make them site collection administrators! You are doing this so that you know who potentially will need the information, and when they would need it (if you want to pass on monthly reports concerning site sizes).
  4. Work in a growth rate monitoring plan. This is very important. You should be proactive and monitoring sites which appear to be growing faster than expectation. This is with a view to investigating what is causing the site growth and what needs to be done to address any issues.
  5. Ensure there is governance put in place to ensure the site templates are set to a company standard, and that if there are any changes they can be applied easily. Remember that in doing this you are ensuring your site size roadmap connects into Disaster Recovery and site topology. This also paves the way for future migrations of the SharePoint platform.

In conclusion.

Whatever the way you apply your site templates, you must map this to some kind of governance that supports it. Whether that governance is defined by a SharePoint architect on behalf of an organization without yet a SharePoint governance board, or whether technology defines costing for storage, or whether the business can request more space at a drop of a hat, you will still need to manage and monitor the impact.

This article has concentrated on the concept of site quota management. There are of course many other articles describing how to implement and configure site quotas in SharePoint.

Create, edit, and delete quota templates in SharePoint 2013

http://technet.microsoft.com/en-us/library/cc263223.aspx

Managing Site Collection Storage Limits (SharePoint 2010)

http://technet.microsoft.com/en-us/library/cc263480.aspx

Plan Quota Management (SharePoint 2010)

http://technet.microsoft.com/en-us/library/cc891489.aspx

Create, edit and delete quota templates (SharePoint 2010)

http://technet.microsoft.com/en-us/library/cc263223.aspx

 

Can’t Start SharePoint 2010 Sync Service, Error 1001, 8312, 6398 – a Fix?

Can’t Start SharePoint 2010 Sync Service, Error 1001, 8312, 6398 – a Fix?

Best way to start is to show you those errors in the good ole event log:

Error 1001:

Fault bucket , type 0

Event Name: CLR20r3

Response: Not available

Cab Id: 0

Problem signature:

P1: owstimer.exe

P2: 14.0.5128.5000

P3: 4cace1f1

P4: mscorlib

P5: 2.0.0.0

P6: 4bebe8e2

P7: 420f

P8: 51

P9: DI4EQZGVCWKDLTHKQTZWPXE3YRFKN1I2

P10:

Attached files:

These files may be available here:

C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_owstimer.exe_8e20523ad5db1439a93c757fe02d53f596a2ef32_435aa75b

Analysis symbol:

Rechecking for solution: 0

Report Id: 43b091e0-304b-11e0-b81c-005056932bb3

 

Error 8312:

SharePoint Web Services Round Robin Service Load Balancer Event: Initialization

Process Name: OWSTIMER

Process ID: 6160

AppDomain Name: DefaultDomain

AppDomain ID: 1

Service Application Uri: urn:schemas-microsoft-com:sharepoint:service:829500b757c944debc8e4e5da76e5b07#authority=urn:uuid:69453d27abe74895b8385be3dc9c2e35&authority=https://plonapdmps01:32844/Topology/topology.svc

Active Endpoints: 1

Failed Endpoints:0

Endpoint List:

http://sharepointindexserver:32843/829500b757c944debc8e4e5da76e5b07/ProfileService.svc

 

Error 6398:

Log Name: Application

Source: Microsoft-SharePoint Products-SharePoint Foundation

Date: 04/02/2011 01:00:01

Event ID: 6398

Task Category: Timer

Level: Critical

Keywords:

User: SERVICEACCOUNT

Computer: sharepointindexserver.com

Description:

The Execute method of job definition Microsoft.Office.Server.UserProfiles.UserProfileImportJob (ID bee5da2c-0ec8-4dce-9f13-7116fdaa9256) threw an exception. More information is included below.

User profile import is in progress!Import could not be started. Please start import later or stop import now for new import to start.

 

So what’s the problem?

Noticed that when attempting to run Sync jobs that the log was not being updated and a sync job would immediately go to Idle.

Noticed also that no profiles had been updated for over a week.

Tried to check the Sync Connections. Noticed that if Sync is running the Query on Sync Connections shows nothing (most confusing) and the only way you find out that everything is actually ok is to click ADD NEW CONNECTION – actually, I decided to check instead ForeFront in Services and this is what greeted me.

04-02-2011 10-56-07.png

Forefront Identity Manager Service not started? Er – How? WT? 

And the Fix is? Drum Roll…

Seems that the ForeFront Identity Manager had stopped? [Shocked]

So, restarted ForeFront Identity Manager, and restarted the sync jobs.

Result? Success!

Why did the Fix work?

Now ‘why’ – am asking myself – can only imagine that:

1: If the Identify Manager is not working it seemed that no connections were displayed in Sync Connections, and if a new connection is attempted to be created then the Correlation Error MOSS MA was displayed – restarting Identity Manager fixed that.

2: If Forefront is configured to look at the Forest and not a specific domain in that forest then it is possible that the import will stop if the user id does not have access to that domain in the forest

3: Error 1001 showed that the job attempted to start then immediately failed, but did not indicate that ForeFront had anything to do with the job itself. I suppose the synchronisation job failed, but that would mean checking all other related jobs, and not any of the services (unless you decided to check Forefront first – as I did!)

4: Error 8312 showed an issue with the Profile Status service meaning that that job did not start or shutdown prematurely; again seems to relate to ForeFront and the Sync job

5: Finally, good old 6398 – showing that the import process could not be started – I suppose that means a fault in attempting to even start the sync process!

6: I have a very strong suspicion that the installation of the culmulative updates for Dec 2010 SharePoint affected the User Profiles sync and shut down ForeFront Identity Manager. In future for any culmulative update I will check that all those relevant services are running – in fact, a good idea is to note the state of services on the index box before applying the update. I know that sounds a bit over the top, but this kind of error completely threw me and I spent I think far too long attempting to get to the bottom of it!!

So, the upshot? Scratching Head Time but at least the problem has been corrected. So, am still investigating this one, but hope this little nugget helps you guys out there.

Considering and Planning SharePoint 2010 Topology

Considering and Planning SharePoint 2010 Topology

As a SharePoint Architect, it is crucially important that to create a robust SharePoint environment for your clients requires planning and considerations of opportunities and threats concerning SharePoint topologies.

All too often SharePoint is implemented without a true understanding of its premise, structure and components, potentially leading to SharePoint becoming a white elephant from a support and user perspective. However, if planning is carried out, and considered by you, your peers and the client and following a repeatable process then it ensures platform growth and future-proofing.

This article of two discusses my opinions in creating a detailed design for a SharePoint 2010 environment focusing on designing its topology, and covers these areas:

  • The Customer Base
  • Functionality and Availability
  • Capacity
  • Performance
  • Design Constraints.

In addition, we must not forget areas that relate to planning SharePoint 2010 topologies, namely: Organisational Requirements; SLA requirements; Functional requirements and Licensing (but I’ll get around to that in Article 3 J ).

So, where do we start?

First, learn more about the areas that you will need to document as part of the topology planning exercise. Of course, you don’t have to throw your hands up yelling ‘how do I find that?’, since there is a wealth of information out there already – either hosted by Microsoft on TechNet or through other excellent sources (including my book J)

First – my book is the source for describing how to best implement SharePoint 2010 (everything and much, much more is in that book) – since there are significant pieces of work leading to designing your SharePoint environment:

Managing and Implementation of Microsoft SharePoint 2010 Projects – check out system specification, topology design and user requirements gathering.

Other good reads:

 

Also, it is important to be aware of the components that make up a SharePoint environment. And, the best way to appreciate and gather these properly is to start from the lowest level component; working your way up through increasingly complex and costly areas so a clear idea is known. For those used to SharePoint, these should come as no surprise – in order then:

Lists and Libraries > Sites > Site Collections > Content Databases > Web Applications > Servers > Farms > Datacentres.

Each of these areas will constrain your topology design for SharePoint and must be investigated.

So how do you proceed?

First, document the customer base. Identify the number of users, the level of information held in their user profiles, how those profiles are stored, the physical location of the users and who manages the user profiles.

Second, gather the functional and availability requirements. What is the premise of the SharePoint environment – is it document management? Publishing? Application Workflow? Social Networking? A combination? Will it include Search and to what extent? Will users have personal spaces (Mysites) and if so what basic functionality will they require? Availability – if the SharePoint environment was subject to a disaster event, how long could the business afford to be without SharePoint? If a site became corrupted and had to be restored, how quickly do users need to be up and running with the restored environment? When SharePoint requires maintenance, when are these ‘windows of opportunity’ available and for how long?

Thirdly, plan capacity. Starting from the base, analyse document storage. This means sizing up the document type and sizing. Do this for all the document requirements for all the relevant sites gathered in the previous step, then, total. Factor in search requirements since this will give you a rough size of the search index, and include all service applications needed on day one (since that will also have a bearing on the content data sizes as well). Include storage requirements for the operating system. Other areas include size backups staging requirements (e.g. direct site backups and farm backups), logs and storage requirements for tools (e.g. third party applications).

Don’t forget that the storage type being used has a bearing since there is an associated cost – ensure that you discuss options with your storage teams as to the best requirements. Typically, you want fast access to content and maybe slower access to archival material so make sure you have choices set for those.

Again, going back to the components that make up the SharePoint environment, confirm each of them against the above and be prepared to go to the n’th degree to ensure you are clear on whether the planning meets the provision and limitations of those components. A very good place to understand these is here: http://technet.microsoft.com/en-us/library/cc262787.aspx

I hope this has been useful to you. The second article of this one will further discuss other key parts of this process, Performance and Design constraints.

How to install Adobe PDF Filtering for SharePoint 2010

How to install Adobe PDF Filtering for SharePoint 2010

This article explains how to install the PDF filtering ability in SharePoint 2010. As you may be aware, without this ability text within PDF files will not be included in any SharePoint index, and as such, only the PDF filename itself will be indexed.

As always with any third party application, make sure you deploy this first to a test platform, confirm it works through tests, and then through change process apply it to your production environment if successful.

Steps.

1: Download the Adobe Filtering Package – called the Adobe PDF 64bit iFilter (at version 9 at the point of writing this document) from here:

http://www.adobe.com/support/downloads/detail.jsp?ftpID=4025

2: Extract the PDFiFilter64installer to a location accessible from your SharePoint 2010 Farm.

3: Log into your Application Server

4: Run the PDFiFilter64installer:

 

 

 

 

 

4: Repeat process (3) for all servers in the Farm

5: Log into SharePoint 2010 Central Administration

6: In the Application Management section, click Manage Service Applications

 

7: In the Manage Service Application section, click your Search Service Application

8: In the Search Application, and within the Crawling section, click FILE TYPES

9: The File Types section is displayed. Now to create a PDF file type. Click NEW FILE TYPE

10: Enter .pdf then click OK

11: Use RegEdit to update the Search Filters

Navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Setup\Filters

Right-click the Filters folder and select New Key. Enter “.pdf” for the key value.

Add the following values to this key:

<REG_SZ> Default = <value not set>

<REG_SZ> Extension = pdf

<REG_DWORD> FileTypeBucket = 1

<REG_SZ> MimeTypes = application/pdf

The key should look like this:

12: Use RegEdit to update the Common Filters Extension

Navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Setup\ContentIndexCommon\Filters\Extension

Right-click the Filters folder and select New Key. Enter .pdf for the key value.

Set the “default” value to

{E8978DA6-047F-4E3D-9C78-CDBE46041603}

The key should look like this:

 

13: Find a PDF icon – this should be either a .GIF or .PNG, and copy this into the following location:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\IMAGES

14: Access the following directory:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\XML

15: Copy DOCICON.XML to ZZDOCICON – Copy.XML (to ensure you have a backup copy)

16: Open DOCICON.XML using Notepad

17: Insert a blank line under <Mapping key = “onetoc2”, copy and paste the following into that blank line:

<Mapping Key=”pdf” Value=”pdf_icon.gif”/>

If done correctly that section of DOCICON.XML should look like this:

 

18: Save DOCICON.XML

19: Run IISRESET on the server

20: Restart services SharePoint Foundation Search V4 and SharePoint Server Search 1

21: Repeat steps 11 through to 20

That completes the process of installing the PDF iFilter for SharePoint 2010; run a re-crawl or wait for the next scheduled to start seeing your PDFs being indexed.

Note.

  • Make sure your PDF entries in REGEDIT is absolutely correct on all the servers where it is installed. This accounts for virtually all the reasons why PDFs do not appear indexed.
  • If the PDF icon does not appear it may be because the image is invalid, corrupted or simply cannot be displayed.
Can’t Start SharePoint 2010 Sync Service, Error 1001, 8312, 6398 – a Fix?

User Profiles – Picture Property in SharePoint 2010

First blog of the Year and its good ole Administration in SharePoint 2010 land, starring User Profiles.

Now back in SharePoint 2007 SSP User Profiles, you may have thought that configuration was easy and simplistic. You know, setting a property to a value in AD was a done deal, right?

Let’s take an issue which had me stumped; which I managed to resolve and suspect was down to a combination of calls to Microsoft, heavy trawling of MS blogs, great SharePoint blogs, my own blogs on Service Applications. And of course, starring a mix of several test environments.

First, let’s describe the problem.

Users have their picture thumbnails created and pushed into AD via Third Party. The property that stores the picture is called ThumbNailPhoto. We need to display that photo in the search results when doing a People Search, and also in their Mysite when the user is being displayed. We don’t allow the users to change the photo so we only want to import them.

Describing the platforms I worked on

1: A three server SharePoint 2010 farm connected to its own AD copy of 2000 users.

2: A single server SharePoitn 2010 farm connected to its own AD copy of 1000 users.

Each platform has a number of photos uploaded to named accounts, called SPTest1 through to 100.

Description of the problem

Now, if the picture thumbnail was a URL pointing to a location of the user and the property in SharePoint being used to show the Picture was also a URL then – yep, this would be easy lets map the PictureURL to the PictureURL. However, that wont help in SharePoint 2010. Why? Well, the main reason for this is quite simply that the PictureURL property is a URL field (storing only the HTTP address pointing to a picture file), whereas AD stores the entire image itself as a “binary large object” (blob). So any synchronisation will involve downloading the pictures stored in AD, putting them somewhere HTTP-accessible, and then setting the PictureURL user profile property to the address of this copied image file.

Additionally, there was no out-of-box support for synchronizing user profile pictures from Active Directory to SharePoint (SharePoint to AD worked) in 2010.

Here’s what I tried before getting to stumped level.

1: Attempted to set the import Picture URL property to ThumbnailPhoto – recrawled the people search, nothing.

2: Attempted to set the export Picture URL property to ThumbnailPhoto and put a picture into my MYSITE user profile – hey that worked!

3: Tried the same on several test farms before building two more farms (as described above) and applying the October CU – several failures because I could not get the sequence right – see below.

Following an investigation further and discussions with other SharePoint Architects with the exact same problem, I found out that the issue had been escalated to Microsoft.

Microsoft provided the 2010 October CU hotfix; then removed and then re-released release. One of the ‘fixes’ was to provide the ability for the import of thumbnails from AD to SharePoint. Unfortunately, I found that a number of steps was still required to get photos to be displayed.

Steps.

Before continuing, let me state that the following steps I’ve repeated four times on separate SharePoint instances and each one has worked. If they don’t for you check the version of your farm to ensure that the October CU has taken hold. Make sure also that the CU has completely deployed. Also, make sure that you do have accounts where the ThumbnailPhoto property has a picture assigned to it. I know that might sound completely daft but I’ve found examples where the admins are saying ‘It doesn’t work’ when in fact no pictures have been even set in the relevant property.

Let’s begin.

1: Deploy the October CU to all of the servers in the farm. This means installing it in full, and as you do note that the servers must be rebooted. I installed the CU on the Index Servers and then the web front ends, by letting the installation take place on all three, then rebooting the index server first and running the Products Configuration Wizard on the index server, then repeating that process for each of the front end servers. Then, when installed on the application server ensure that the Profile Sync Service is running (Central Administration, Services on Server, User Profile Synchronization Service).

2: Update the ThumbnailPhoto Property. Access your User Profile Application on the CA, go to Manage User Properties, access the Picture Property, then map the Picture Property to your Active Directory ThumbnailPhoto Property

3: Run a User Synchronization job to re-populate the user profiles.

4: Once the User Synchronization job is complete, access the user properties to see if the Picture Property has picked up some profiles. If it has you’re nearly there!

5: Run a Powershell script to update the profile pictures to the MySite URL. Open up the SharePoint Management Shell and run the following command:

Update-SPProfilePhotoStore -CreateThumbnailsForImportedPhotos 1 -MySiteHostLocation http://<YourServerName>/my

This ensures the profile photos are set into the relevant MySite so that when the user is located their picture is displayed. Additionally, I’ve found that even if you run a sync job to get the pictures from AD that the number of profiles against the property is not updated. The above command corrects that issue.

6: Finally, run the People Crawl to update the user search properties so that when searching for the user the images are displayed.

That’s all there is to it! Hope this helps out you guys out there.