Summary
A customer asked if there was a method to identity documents stored in SharePoint online that were encrypted with passwords. Since nothing like this existed, it was created using PowerShell. I’m sharing this because the logic in the script may be useful for others.
The Code
#Title: find-docpasswords
#Description: Iterates through each item in a specified list to find documents stored with passwords.
#Date: 7/8/2020
#Author: Mike Lee
#Disclaimer: This PowerShell script is provided "as-is" with no warranties expressed or implied. Use it at your own risk.
#Dependencies: SharePoint Online Client Components SDK: https://www.microsoft.com/en-us/download/details.aspx?id=42038
#Tested with SharePoint Online Client Components SDK version 16.0.6906.1200
#Parameters: $SiteURL, $ListName, $username
#Add references to SharePoint client assemblies
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
[System.Reflection.Assembly]::LoadWithPartialName("WindowsBase")
#Your SPO Tenant
$SiteURL = "https://tenant.sharepoint.com"
#The name of your document library
$Listname = "Documents"
#The admin account that has access to the library
$username = "admin@tenant.onmicrosoft.com"
$password = Read-Host "Enter Password" -AsSecureString
#Building Context
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userName, $password)
$List = $ctx.Web.Lists.GetByTitle($ListName)
#CAML Query to recursively look at all items in the library with a 5000 item row limit.
$camlQuery = New-Object Microsoft.SharePoint.Client.CamlQuery
$camlQuery.ViewXml = @"
<View Scope="RecursiveAll">
<Query>
<OrderBy><FieldRef Name='ID' Ascending='TRUE'/></OrderBy>
</Query>
<RowLimit Paged="TRUE">5000</RowLimit>
</View>
"@
$items = $list.GetItems($camlQuery)
$ctx.Load($items)
$ctx.ExecuteQuery()
#function to read documents
function find-docpasswords($ctx, $FileUrl)
{
#Collect Documents Data
$FileURL = $Item.FieldValues['FileRef']
#Read the files from SharePoint online document library.
$fileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($ctx,$FileURL)
$stream = New-Object System.IO.MemoryStream
$fileInfo.Stream.CopyTo($stream)
#Read the first row of bytes as text
$Start = [System.Text.Encoding]::Default.GetString($stream.ToArray()[0000..2000])
# Record files that are password protected
if($Start -match "E.n.c.r.y.p.t.e.d.P.a.c.k.a.g.e")
{
Write-Host "$SiteURL$FileURL -- Is Password Protected" -ForegroundColor Yellow
}
else
{
Write-Host "$SiteURL$FileURL -- Not Password Protected" -ForegroundColor Green
}
$stream.Close()
$fileinfo.Dispose()
$ctx.Dispose()
}
#Run the function to loop through all items in the library and find documents stored with passwords
foreach($item in $items)
{
$fileUrl = $item.FieldValues["fileref"]
find-docpasswords $ctx $fileurl
}
Takeaways
This scripts loops though a specified document library and reads the first 200 binary bytes as text. If the encrypted string is found, the document URL is reported in the console output.
Here is an example of the output:
You will need a few things to make this works.
- Installed the SharePoint Online Client Components SDK
- Specify the “$SiteURL, $Listname, and $username in the script.
Train your mind and the tech will follow.
This series of training videos covers the value and recent updates across SharePoint, OneDrive, Microsoft Lists, Microsoft Teams, Project Cortex, Yammer, and Stream – including information for employees, admins, and developers – all in the context of scenarios supported within Microsoft 365 across teamwork and employee engagement.
It is all here in one blog post, inline… consume at your leisure.
Scroll down and learn about all as you go or click each title in the grid below to jump down the page to watch specific sessions – each with related links to learn more:
Microsoft 365 Collaboration [keynote]
Microsoft 365 empowers individuals, teams, and organizations to be creative, collaborative, and effective with an integrated suite of experiences that are simple, superior, smart, and secure. Jeff Teper shares the latest innovations and updates for content collaboration, employee engagement and communications, and knowledge management. Learn how the experiences in Microsoft 365—including SharePoint, OneDrive, Yammer, Stream, and Office—integrate to power collaboration across devices, on the web, in desktop and mobile apps, and in the hub for teamwork, Microsoft Teams.
Presented by Jeff Teper [LinkedIn | Twitter], Omar Shahine [LinkedIn | Twitter], Navjot Virk [LinkedIn | Twitter], Susan Hanley [LinkedIn | Twitter], Graham Sheldon [LinkedIn | Twitter], and Ed Averett [LinkedIn | Twitter].
Learn more:
[Back to top]
OneDrive powers intelligent file experiences across Microsoft 365
OneDrive is the intelligent files app for Microsoft 365. In this session we will cover upcoming innovations and explore functionalities that empower you to access, share and collaborate on all your files from anywhere while protecting your work.
Presented by Randy Wong [LinkedIn | Twitter].
Learn more about OneDrive in Microsoft 365: https://aka.ms/OneDrive/blog
[Back to top]
Collaboration & external file sharing across Microsoft 365
Microsoft 365 provides a rich set of solutions for collaborating with users both inside and outside of your organization. This session offers an in-depth look at existing and brand-new external sharing capabilities. Learn best practices for configuring external sharing and educating users on how to best leverage Teams, SharePoint, and OneDrive for collaborating with others.
Presented by Ankita Kirti [LinkedIn | Twitter].
Learn more:
[Back to top]
Microsoft Lists – Share and track information with across Microsoft 365
Get an early look at Microsoft Lists – your smart information tracking app in Microsoft 365. Lincoln will showcase how Lists evolve from SharePoint lists today and how current and new innovation empower individuals and teams to create, share and track information – all in the apps they use every day, including Microsoft Teams. Lots of demos highlighting Lists home, mobile, conditional formatting, fast quick edit, ready-made templates, new views, alerts, flows and more – work tracking with built-in security and compliance.
Presented by Lincoln DeMaris [LinkedIn | Twitter].
Learn more:
[Back to top]
Design productivity apps with SharePoint lists and libraries, Power Apps, and Power Automate
No-code and low-code apps have been essential in SharePoint for a long time. Additional tolls from the Power Platform – Power Apps and Power Automate – provide additional power and capability to build productivity using SharePoint lists as the data source. We also discuss the patterns to help transform customer solutions that remain on-premises and/or in legacy tool sets like InfoPath, SharePoint Designer or Access Web Apps.
Presented by Chakkaradeep “Chaks” Chandran [LinkedIn | Twitter].
Learn more how to create a Power App for a list in SharePoint for Microsoft 365.
[Back to top]
Knowledge and Project Cortex – the Microsoft 365 Vision
In a world of rapid change, harnessing knowledge empowers people to act quickly, and organizations to be more resilient. Later this year, Microsoft will release Project Cortex, our new knowledge and content management solution. Join us for an overview of Project Cortex and Microsoft Search and how they help customers harness knowledge throughout Microsoft 365.
Presented by Naomi Moneypenny [LinkedIn | Twitter] and Chris McNulty [LinkedIn | Twitter].
Learn more about Project Cortex: https://aka.ms/ProjectCortex.
[Back to top]
Connect the workplace with engaging, dynamic experiences across your intranet
The intelligent intranet in Microsoft 365 powers collaboration, employee engagement, and knowledge management. The intelligent intranet is mobile-ready, personalized, social, and actionable. Join in and explore innovations like multilingual pages, navigation, news, pages, and broader integrated solutions with Yammer, Stream, and Microsoft Teams.
Presented by Debjani Mitra [LinkedIn | Twitter] and Brad McCabe [LinkedIn].
Learn more:
[Back to top]
The New Yammer
The new Yammer is landing. See how Yammer has been completely redesigned to power leadership engagement, company-wide communication, and communities in Microsoft 365. Explore the new capabilities, features, and styling while discovering how Yammer continues to supercharge community, knowledge sharing, and engagement across Microsoft 365.
Presented by Jason Mayans [LinkedIn | Twitter].
Learn more about The New Yammer and Yammer in general.
[Back to top]
Microsoft 365 Live Events and remote work
Learn how live events can help your organization deliver better communications, training and more. In this session, we will walk through a detailed setup of a live event and look at the underlying technology that enables successful live broadcasts.
Presented by Lorena Huang Liu [LinkedIn | Twitter] & Christina Torok [LinkedIn | Twitter].
Learn more about Microsoft Live Events.
[Back to top]
Architecting Your Intranet
You are ready to start bringing the power of the intelligent intranet to your organization, but you are wondering where to start. What should you do with your existing sites? What about navigation? How should you think about your IA? These are just a few of the questions you might be wondering. Join us for this discussion as we help break through the analysis paralysis and learn from the best practices of numerous customers that have already started the journey.
Presented by Melissa Torres [LinkedIn | Twitter].
Learn more about the Microsoft 365 intelligent intranet and how to use SharePoint site designs and site scripting in Microsoft 365.
[Back to top]
Migration to SharePoint, OneDrive, and Microsoft Teams in Microsoft 365, free and easy
Regardless of your organization’s size, data scale or information complexity, you can migrate documents and sites into SharePoint in Office 365 successfully. Come learn about new capabilities available in the SharePoint Migration Tool (aka.ms/SPMT) in addition to performance and reliability investments to best assess, plan and implement your migration. Learn how to migrate file shares, doc libraries, and SharePoint Server 2013 sites and more using the SharePoint Migration Tool and new capabilities to simplify large file share migrations through SharePoint Admin Center.
Presented by Hani Loza [LinkedIn | Twitter] and Eric Warnke [LinkedIn | Twitter].
Learn more:
[Back to top]
SharePoint developer overview
Discover how every experience can become more collaborative and engaging with the Microsoft 365 platform – starting with SharePoint. SharePoint serves as the center of collaboration – where content is stored and communicated across teams, departments, and the whole organization. See the latest development advancements and new capabilities for SharePoint. You’ll take away new ideas for building next-generation collaboration applications and get the essential roadmap for custom apps in your organization.
Presented by Luca Bandinelli [LinkedIn].
Learn more about the SharePoint Framework (SPFx).
[Back to top]
Jumpstart your projects with community projects from Patterns and Practices (PnP)
Across SharePoint and Microsoft 365, an extremely active developer community has come together with SharePoint engineering and have released numerous valuable reusable components and controls, which will simplify design, implementation, and management of Microsoft 365 and on-premises deployments. Join this session to hear the latest updates around the different guidance, samples, and reusable assets built by the community for the community.
Presented by Vesa Juvonen [LinkedIn | Twitter].
Learn more:
[Back to top]
Security and compliance in SharePoint and OneDrive
Safeguard your devices, personal information, and files from being compromised. This session explores the core tenets of platform security, secure access and sharing, information governance, and compliance across SharePoint, Microsoft OneDrive, and Microsoft 365.
Presented by Sesha Mani [LinkedIn | Twitter].
Learn more about security and compliance in Microsoft 365.
[Back to top]
Well, if you’ve made it this far then your mind is full. And now, the tech shall follow.
Thanks for learning with us :),
Mark
What is SharePoint Perfwiz and why do we need a replacement?
SharePoint Perfwiz was a tool used by Product Support to create performance log counters on SharePoint servers to troubleshoot performance issues. This tool has been deprecated but there is still a need to collect performance data for those pesky performance issues.
The Perfwiz tool simply used LOGMAN.exe to create a custom counter set with several selected counters specifically for SharePoint servers.
This blog will detail how to use LOGMAN.exe to mimic the baseline counter set that was created by the Perfwiz tool.
The command Line
Here is the command used to create the Baseline counter set:
Logman.exe create counter Baseline_Counters -o “c:perflogsBaseline_Counters.blg” -f bincirc -v mmddhhmm -max 250 -c “.NET CLR Exceptions(*)*” “.NET CLR Loading(*)*” “.NET CLR Memory(*)*” “.NET CLR Networking(*)*” “.NET Data Provider for SqlServer(*)*” “AppFabric Caching:Host(*)*” “ASP.NET Apps v2.0.50727(*)*” “ASP.NET Apps v4.0.30319(*)*” “ASP.NET v2.0.50727*” “ASP.NET v4.0.30319*” “ASP.NET(*)*” “LogicalDisk(*)*” “Memory*” “NBT Connection(*)*” “NetLogon(*)*” “Network Interface(*)*” “Office Web Apps – Online Viewing*” “PhysicalDisk(*)*” “Process(*)*” “Processor Information(*)*” “Processor(*)*” “Sandboxed Code Process Pool(*)*” “Search Flow Statistics(*)*” “Search Host Controller(*)*” “Search Linguistics(*)*” “Search Platform Services(*)*” “Search Query Processing(*)*” “Search Query Processor – SharePointServerSearch(*)*” “Search SPLookupService(*)*” “Server Work Queues(*)*” “Server*” “Shared Service Provider(*)*” “SharePoint Disk-Based Cache(*)*” “Sharepoint Distributed Cache Counters(*)*” “SharePoint Foundation (*)*” “SharePoint Foundation Authentication (*)*” “SharePoint Foundation BDC Metadata*” “SharePoint Foundation BDC Online(*)*” “SharePoint Foundation Request Management(*)*” “SharePoint Foundation Security Token Service*” “SharePoint Publishing Cache(*)*” “SharePoint Server Cache instances(*)*” “SharePoint Server Cache*” “SiteComponents*” “System*” “TCPv4*” “TCPV6*” “Thread(w3wp_*)*” “Thread(OWSTIMER_*)*” “W3SVC_W3WP(*)*” “WAS_W3WP(*)*” “Web Service Cache*” “Web Service(*)*” “Windows Workflow Foundation(*)*” -si 00:00:30 -cnf 12:00:00
Switches used:
-o: The output file
-f: Sets the file to a circular binary
-v: Adds a date / time stamp in the file name
-max: The max file size
-c: The counter list
-si: The sample interval
-cnf: 12:00:00 tells the counter to create a new file and continue when the max size has been reached or after 12 hours.
What is looks like
To run the LOGMAN.exe command, you will need an elevated command prompt to run this command.
Here is what it looks like after the counters are created:
Here is an example of the output file:
After the counters are created, they can be started with the following command:
logman start Baseline_Counters
After the problem is reproduced, stop the counters with the following command:
logman stop Baseline_Counters
However, they can be started and stopped manually as well.
Important notes and takeaways
- If you have multiple servers, you can use the -s switch and feed in a server list, example:
$servers = Get-Content c:tempservers.txt
foreach ($server in $servers) {
Logman.exe create counter -s $server 'Baseline_Counters' -o 'c:perflogsBaseline_Counters.blg' -f bincirc -v mmddhhmm -max 250 -c '".NET CLR Exceptions(*)*" ".NET CLR Loading(*)*" ".NET CLR Memory(*)*" ".NET CLR Networking(*)*" ".NET Data Provider for SqlServer(*)*" "AppFabric Caching:Host(*)*" "ASP.NET Apps v2.0.50727(*)*" "ASP.NET Apps v4.0.30319(*)*" "ASP.NET v2.0.50727*" "ASP.NET v4.0.30319*" "ASP.NET(*)*" "LogicalDisk(*)*" "Memory*" "NBT Connection(*)*" "NetLogon(*)*" "Network Interface(*)*" "Office Web Apps - Online Viewing*" "PhysicalDisk(*)*" "Process(*)*" "Processor Information(*)*" "Processor(*)*" "Sandboxed Code Process Pool(*)*" "Search Flow Statistics(*)*" "Search Host Controller(*)*" "Search Linguistics(*)*" "Search Platform Services(*)*" "Search Query Processing(*)*" "Search Query Processor - SharePointServerSearch(*)*" "Search SPLookupService(*)*" "Server Work Queues(*)*" "Server*" "Shared Service Provider(*)*" "SharePoint Disk-Based Cache(*)*" "Sharepoint Distributed Cache Counters(*)*" "SharePoint Foundation (*)*" "SharePoint Foundation Authentication (*)*" "SharePoint Foundation BDC Metadata*" "SharePoint Foundation BDC Online(*)*" "SharePoint Foundation Request Management(*)*" "SharePoint Foundation Security Token Service*" "SharePoint Publishing Cache(*)*" "SharePoint Server Cache instances(*)*" "SharePoint Server Cache*" "SiteComponents*" "System*" "TCPv4*" "TCPV6*" "Thread(w3wp_*)*" "Thread(OWSTIMER_*)*" "W3SVC_W3WP(*)*" "WAS_W3WP(*)*" "Web Service Cache*" "Web Service(*)*" "Windows Workflow Foundation(*)*"' -si 00:00:30 -cnf 12:00:00
}
- The default action of this counter set is to run forever, you will need to stop them manually once this data is no longer needed.
In this weekly discussion of latest news and topics around Microsoft 365, hosts – Vesa Juvonen (Microsoft), Waldek Mastykarz (Rencore), are joined by Paul Bullock – SharePoint Architect and MVP with CaPa Creative located in the UK. Paul is a major contributor to PnP modernization tooling effort which leads to this call’s discussion focus.
- So why would you share your code as open-source?
- When’s a good time to start contributing?
- How do you get plugged into the PnP community?
This session is a great place to start. PnP is not just code, it’s structure, infrastructure, policy, adoption strategies, recognition, networking, the human-side of IT. Open-source is great way to learn from and work with people who are introverts, extroverts, people located near and far with various organization affiliations, customer projects and technical skills that share common passions and a relationship to the PnP community. Additionally, in this episode, 17 recently released articles from Microsoft and the PnP Community are highlighted.
As always, if you need help on an issue, want to share a discovery, or just want to say: “Job well done”, please reach out to Vesa, to Waldek or to your PnP Community.
This episode was recorded on Monday, June 29, 2020.
Did we miss your article? Please use #PnPWeekly hashtag in the Twitter for letting us know the content which you have created.
On March 24th we shared in an announcement in the M365 message center (MC207439) details around temporary adjustments we are making to select capabilities in SharePoint Online and OneDrive.
During these unprecedented times, we are taking steps to ensure that SharePoint Online and OneDrive services remain available and reliable for your users who depend on the service more than ever in remote work scenarios.
Note: These temporary feature adjustments may be in place during business hours in your tenant's region.
Content migration, Data Loss Prevention (DLP), and backup solutions
Many SharePoint Online and OneDrive customers run business-critical applications against the service that run in the background. These include content migration, Data Loss Prevention (DLP), apps that scan the service and backup solutions. In support of the objective to remain highly available, we are moving some operations to regional evening and weekend hours.
Users may observe:
- Migration, DLP and backup solutions may achieve limited throughput during regional weekday daytime hours. During evening and weekend hours for the region, the service will be ready to process a significantly higher volume of requests from background apps.
Please review the relevant best practice guidance, which describes how to get maximize throughput.
File Management
Various background processes to manage new media (images, videos) may now be processed during evening and weekend hours.
Modified processes include:
- Users may experience reduced video resolution for playback videos.
-
Customers who use OneDrive Files On-Demand and choose to “display items by using large thumbnails” in Windows Explorer or Mac Finder will see generic icons instead of thumbnails.
Note: Photo file thumbnails (jpeg, jpg, png, etc) are not affected by this change.
The affected thumbnail types are categorized as the following:
- Video and PDF files: pdf, avi, mp4, mov, mpg, etc.
- Document files with generators: docx, txt, html, etc.
- New extensions likely to be permanently blocked because no thumbnail generator is ever likely to exist: reg|bak|iso|nupkg, etc.
- Files types already blocked today because no generator exists: lnk|xlsx|xls|url|exe|zip|rar|rdp|apprefms|msi|website
There is no workaround, we will continue listening to feedback and iterate on this approach.
Additional Information
We will provide further updates to this post as the situation may change.
Last article update: 07/1/20
In this weekly discussion of latest news and topics around Microsoft 365, hosts – Vesa Juvonen (Microsoft), Waldek Mastykarz (Rencore), are joined by – Andrew Connell (A.C.) – MVP, Instructor, owner of Voitanos located in Florida, US.
Topics included:
- Andrew’s “Mastering SharePoint Framework” course – a 2-year endeavor that is content complete – well almost.
- Waiting on SPFx v1.11 release.
- Andrew shares his honest opinions on SPFx – capabilities, reliability, completeness, engineering communications, need for functional consistency across apps and tools that encompass more just SharePoint now.
- Discussed using library components or npm packages
- UX components – using Office UI Fabric or Fluent Fabric.
- Additionally, in this episode, 18 recently released articles from Microsoft and the PnP Community are highlighted.
As always, if you need help on an issue, want to share a discovery, or just want to say: “Job well done”, please reach out to Vesa, to Waldek or to your PnP Community.
This episode was recorded on Monday, June 22, 2020
Did we miss your article? Please use #PnPWeekly hashtag in the Twitter for letting us know the content which you have created.
Safely sharing and accessing content is becoming increasingly important as the business world shifts to remote work. Join the OneDrive team on June 30, at 9:00-10:00 AM PT for a free webinar that demonstrates how Microsoft 365, OneDrive and SharePoint help users stay productive, keep your data secure and private, reduce the stress on IT during compliance or litigation issues while giving admins the tools to manage and monitor content.
This session is followed by an “Ask Microsoft Anything” session (10:00-11:00 AM PT), where you can bring your questions and feedback to: https://aka.ms/OneDriveAMA
Find all event details here.
In this weekly discussion of latest news and topics around Microsoft 365, hosts – Vesa Juvonen (Microsoft), Waldek Mastykarz (Rencore), are joined by this week are 2 members of the PnP Team and MVPs – Paolo Pialorsi, Consultant and owner PiaSys, Brescia, Italy, and David Warner, Principal Consultant with Catapult Systems, Los Angeles, US.
The discussion topic for the day: How does the PnP team get so much done? It seems like we do a lot because our work in done in the open. Yes, sharing with the PnP Community is indeed a conscious priority after family.
Discussion takeaways: Caring requires contributors to be consistent, to communicate, to be good time managers, to not be self-critical or perfectionists, to be willing to distribute the load, and to find ways to share in a way that benefits your employer, your client and your community at the same time.
“Shipped is better than perfect”, and any size contribution is a welcome contribution. Sharing is caring. Additionally, in this episode, 15 recently released articles from Microsoft and the PnP Community are highlighted.
As always, if you need help on an issue, want to share a discovery, or just want to say: “Job well done”, please reach out to Vesa, to Waldek or to your PnP Community.
This episode was recorded on Monday, June 8, 2020
Did we miss your article? Please use #PnPWeekly hashtag in the Twitter for letting us know the content which you have created.
When a working on a new confidential project, you need to make sure that collaboration (inside and outside your organization) is secured.
in this short 12 minutes video we walk you through the process of creating new sensitive information type, creating a new sensitivity label, configuring SPO and Teams site as well as configuring an Insider Risk policy.
Attached to this post is the video.
This is the first in a series of videos that we are releasing in order to help our customers understand how they can protect their sensitive information using Microsoft 365 tools.
![eDiscovery for Teams Webinar]()
The Advanced eDiscovery solution in Microsoft 365 builds on the existing eDiscovery and analytics capabilities in Office 365. This new solution, called Advanced eDiscovery, provides an end-to-end workflow to preserve, collect, review, analyze, and export content that’s responsive to your organization’s internal and external investigations. It also lets legal teams manage the entire legal hold notification workflow to communicate with custodians involved in a case.
References:
This webinar was presented on Tue May 14th, 2020, and the recording can be found here.
Attached to this post are:
- The FAQ document that summarizes the questions and answers that came up over the course of both Webinars; and
- A PDF copy of the presentation.
Thanks to those of you who participated during the two sessions and if you haven’t already, don’t forget to check out our resources available on the Tech Community.
Thanks!
@Adam Bell on behalf of the MIP and Compliance CXE team
