Panopto API 103 – Organise course folders into subfolders

Return to the Panopto API index page

This piece of code is the first half of why I started learning the Panopto API. This blog post was written after page 204, but is only to do with folders so I’ve called it 103.

Our Panopto server is connected to Blackboard, and the in-built integration is really fantastic. Nearly everything is automatic and all videos are nicely secured to the course in which they were recorded.

Now that the University owns Unison the requirement “secured to the course in which they were recorded” is a little too strict. Videos are normally uploaded to a course, not a course in a particular year. Guest lecturers are normally recorded once and re-used year on year. What we need is the ability to have a lecture capture folder that just has that years recordings, but to also have a course folder that can be re-used without breaking year on year.

… and of course this has to happen without our staff having to do it manually 500 times per year.

What I’m trying to do is probably best explained in a video.
http://coursecast.soton.ac.uk/Panopto/Pages/Viewer/Default.aspx?id=cd46bb0a-3b20-490d-9ac3-2d6babecd4f9 (1 minute)

Warning: This code should not be run on its own. Parent folders must have the permissions of the subfolders or the users will not see it.

The code first part of the code works in a similar way to page 102. The system goes through all of the folders and uses the code from ### to work out how many pages there are and to loop through them all.

I’ve decided that the easiest way to work with all the folders is to make an array up with all the folder names and the matching Guids. working with Folders has a great deal of extra metadata. Making a new Dictionary means we only have to work with two things.

When you define a dictionary you select a key (that needs to be unique, so the Folder Guid works well) and a value (that’s the folder name). For this task we’ll be taking a Blackboard folder

ARCH1001-12345-13-14

and making a course folder

ARCH1001

, which the Blackboard course sits in, and a subject folder

ARCH

that the course folder sits in.

So to work with all those folders (because only the first time we run the code with the subject and course folders not exist) we need to define the dictionaries

Dictionary<Guid, string> allRootFoldersDictionary = new Dictionary<Guid, string>();
Dictionary<Guid, string> subjectFoldersDictionary = new Dictionary<Guid, string>();
Dictionary<Guid, string> courseFoldersDictionary = new Dictionary<Guid, string>();

We also need to define how many results per page we will deal with. You should experiment with your own system by using the commands

bool lastPage = false;
int resultsPerPage = 10;
int page = 0;

and

var watch1 = System.Diagnostics.Stopwatch.StartNew();

...

watch1.Stop();
Console.WriteLine("All folders cycled through in {0} ms", watch1.ElapsedMilliseconds);

to time how long the code takes to run. On my system

5 results per page took 80 seconds
50 results per page took 31 seconds
150 results per page took 21 seconds
250 results per page crashed as it returned more than 2MB of data (which we set on page ###)

We loop through the code, and for each folder we use some RegEx to see what the folder is called.

On my system I know that 4 capital letters is a course folder, 4 capital letters followed by 4 numbers is a subject folder and 4 capital letters followed by 4 numbers followed by more characters is a Blackboard course.

So after checking each folder we put it into the corresponding dictionary.

while (!lastPage)
{
	PanoptoSessionManagement.Pagination pagination = new PanoptoSessionManagement.Pagination { MaxNumberResults = resultsPerPage, PageNumber = page };
	ListFoldersResponse response = sessionMgr.GetFoldersList(sessionAuthInfo, new ListFoldersRequest { Pagination = pagination, SortIncreasing = true }, null);

	if (resultsPerPage * (page + 1) >= response.TotalNumberResults)
	{
		lastPage = true;
		Console.WriteLine("There are {0} results, at {1} results per page and on page {2} means we are on the last page",
			response.TotalNumberResults, resultsPerPage, page + 1);
	}
	else
	{
		Console.WriteLine("There are {0} results, at {1} results per page and on page {2} means there are more pages to loop through",
			response.TotalNumberResults, resultsPerPage, page + 1);
	}

	if (response.Results.Length > 0)
	{
		foreach (Folder folder in response.Results)
		{
			if (Regex.IsMatch(folder.Name, "^[A-Z]{4}$"))
			{
				subjectFoldersDictionary.Add(folder.Id, folder.Name);
				Console.WriteLine("{0} is a subject folder", folder.Name);
			}
			else if (Regex.IsMatch(folder.Name, "^[A-Z]{4}[0-9]{4}$"))
			{
				courseFoldersDictionary.Add(folder.Id, folder.Name);
				Console.WriteLine("{0} is a course folder", folder.Name);
			}
			else if (Regex.IsMatch(folder.Name, "[A-Z]{4}[0-9]{4}.+$") && folder.ParentFolder.Equals(null))
			{
				allRootFoldersDictionary.Add(folder.Id, folder.Name);
			}
			else
			{
				Console.WriteLine("Not adding {0} to all folders", folder.Name);
			}
		}
	}
	else
	{
		Console.WriteLine("No Folders found");
	}

	Console.WriteLine("allFoldersDictionary contains {0} lines", allRootFoldersDictionary.Count);

	page++;

}

So now we have a dictionary containing all the Blackboard folders, then two for the subject and course codes.

We loop through the Blackboard folders…

For ARCH1001-12345-13-14 we first ask “Is there an ARCH folder?”. To do this we ask if the subject array contains “ARCH”.

If it does then we remember the Guid for the folder, we’ll need it in a minute. If it doesn’t then we need to make a new folder. When a new folder is made it returns a Folder object, so we can use that to find the Guid of the new “ARCH” folder.

foreach (var key in allRootFoldersDictionary.Keys)
{
	if (allRootFoldersDictionary[key].Length > 8)
	{
		string subject = allRootFoldersDictionary[key].Substring(0, 4);
		string course = allRootFoldersDictionary[key].Substring(0, 8);
		Guid subjectFolder = Guid.Empty;
		Guid courseFolder = Guid.Empty;

		if (subjectFoldersDictionary.ContainsValue(subject))
		{
			subjectFolder = subjectFoldersDictionary.FirstOrDefault(x => x.Value == subject).Key;
			Console.WriteLine("The subject code {0} already exists with ID {1}", subject, subjectFolder);
			
		}
		else
		{
			Folder newFolder = sessionMgr.AddFolder(sessionAuthInfo,
				allRootFoldersDictionary[key].Substring(0, 4), null, false);
			subjectFoldersDictionary.Add(newFolder.Id, newFolder.Name);
			subjectFolder = newFolder.Id;
			Console.WriteLine("The subject code {0} didn't exist. New folder {1} created in top level folder", subject, newFolder.Id);
		}

Next we do almost exactly the same thing, except this time we are looking for ARCH1001. If we don’t find it in the dictionary we need to make the new folder with a parent folder of the subject above.

If the folder did exist we need to update the parent folder

if (courseFoldersDictionary.ContainsValue(course))
{
	courseFolder = courseFoldersDictionary.FirstOrDefault(x => x.Value == course).Key;
	Console.WriteLine("The course code {0} already exists with ID {1}", course, courseFolder);
	
}
else
{
	Folder newFolder = sessionMgr.AddFolder(sessionAuthInfo,
		allRootFoldersDictionary[key].Substring(0, 8), subjectFolder, false);
	courseFoldersDictionary.Add(newFolder.Id, newFolder.Name);
	courseFolder = newFolder.Id;
	Console.WriteLine("The course code {0} didn't exist. New folder {1} created in parent folder {2}", course, newFolder.Id, subjectFolder);
}

Finally we know that ARCH1001-12345-13-14 exists, it’s the folder we are dealing with in the loop, so we need to update its parent folder to be what we found (or made) above.

sessionMgr.UpdateFolderParent(sessionAuthInfo, key, courseFolder);
Console.WriteLine("Updating {0} into {1}", allRootFoldersDictionary[key], courseFolder);

Then we just carry on the loop a couple of hundred times.

I recorded the process of running this code in this video
http://coursecast.soton.ac.uk/Panopto/Pages/Viewer/Default.aspx?id=97a5d82e-59fd-44aa-8fba-7e5fef18cdaa (5 minutes)

Full code below

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Text.RegularExpressions;
using ConsolePanoptoAPI.PanoptoAccessManagement;
using ConsolePanoptoAPI.PanoptoAuth;
using ConsolePanoptoAPI.PanoptoRemoteRecorderManagement;
using ConsolePanoptoAPI.PanoptoSessionManagement;
using ConsolePanoptoAPI.PanoptoUsageReporting;
using ConsolePanoptoAPI.PanoptoUserManagement;

// ANY NEW NAMESPACES SHOULD BE ADDED HERE

// END OF NAMESPACES

namespace ConsolePanoptoAPI
{
    class Program
    {
        static bool hasBeenInitialized = false;

        static void Main(string[] args)
        {
            // PUT YOUR AUTHENTICATION DETAILS HERE
            PanoptoSessionManagement.AuthenticationInfo sessionAuthInfo = new PanoptoSessionManagement.AuthenticationInfo()
            {
                UserKey = "api",
                Password = "s2ezupajePhasaP5"
            };
            // END OF AUTHENTICATION DETAILS

            EnsureCertificateValidation();

            // START WRITING CODE HERE

            bool lastPage = false;
            int resultsPerPage = 150;
            int page = 0;

            /*
             * Test on time taken (ms) to loop through pages based on the following results per page
             * 005 folders per page = 79,762
             * 050 folders per page = 30,911
             * 150 folders per page = 21,087
             * 250 folders per page = crash
             */

            Dictionary<Guid, string> allRootFoldersDictionary = new Dictionary<Guid, string>();
            Dictionary<Guid, string> subjectFoldersDictionary = new Dictionary<Guid, string>();
            Dictionary<Guid, string> courseFoldersDictionary = new Dictionary<Guid, string>();

            ISessionManagement sessionMgr = new SessionManagementClient();

            var watch1 = System.Diagnostics.Stopwatch.StartNew();

            while (!lastPage)
            {
                PanoptoSessionManagement.Pagination pagination = new PanoptoSessionManagement.Pagination { MaxNumberResults = resultsPerPage, PageNumber = page };
                ListFoldersResponse response = sessionMgr.GetFoldersList(sessionAuthInfo, new ListFoldersRequest { Pagination = pagination, SortIncreasing = true }, null);

                if (resultsPerPage * (page + 1) >= response.TotalNumberResults)
                {
                    lastPage = true;
                    Console.WriteLine("There are {0} results, at {1} results per page and on page {2} means we are on the last page",
                        response.TotalNumberResults, resultsPerPage, page + 1);
                }
                else
                {
                    Console.WriteLine("There are {0} results, at {1} results per page and on page {2} means there are more pages to loop through",
                        response.TotalNumberResults, resultsPerPage, page + 1);
                }

                if (response.Results.Length > 0)
                {
                    foreach (Folder folder in response.Results)
                    {
                        if (Regex.IsMatch(folder.Name, "^[A-Z]{4}$"))
                        {
                            subjectFoldersDictionary.Add(folder.Id, folder.Name);
                            Console.WriteLine("{0} is a subject folder", folder.Name);
                        }
                        else if (Regex.IsMatch(folder.Name, "^[A-Z]{4}[0-9]{4}$"))
                        {
                            courseFoldersDictionary.Add(folder.Id, folder.Name);
                            Console.WriteLine("{0} is a course folder", folder.Name);
                        }
                        else if (Regex.IsMatch(folder.Name, "[A-Z]{4}[0-9]{4}.+$") && folder.ParentFolder.Equals(null))
                        {
                            allRootFoldersDictionary.Add(folder.Id, folder.Name);
                        }
                        else
                        {
                            Console.WriteLine("Not adding {0} to all folders", folder.Name);
                        }
                    }
                }
                else
                {
                    Console.WriteLine("No Folders found");
                }

                Console.WriteLine("allFoldersDictionary contains {0} lines", allRootFoldersDictionary.Count);

                page++;

            }

            watch1.Stop();
            Console.WriteLine("All folders cycled through in {0} ms", watch1.ElapsedMilliseconds);

            var watch2 = System.Diagnostics.Stopwatch.StartNew();

            foreach (var key in allRootFoldersDictionary.Keys)
            {
                if (allRootFoldersDictionary[key].Length > 8)
                {
                    string subject = allRootFoldersDictionary[key].Substring(0, 4);
                    string course = allRootFoldersDictionary[key].Substring(0, 8);
                    Guid subjectFolder = Guid.Empty;
                    Guid courseFolder = Guid.Empty;

                    if (subjectFoldersDictionary.ContainsValue(subject))
                    {
                        subjectFolder = subjectFoldersDictionary.FirstOrDefault(x => x.Value == subject).Key;
                        Console.WriteLine("The subject code {0} already exists with ID {1}", subject, subjectFolder);
                        
                    }
                    else
                    {
                        Folder newFolder = sessionMgr.AddFolder(sessionAuthInfo,
                            allRootFoldersDictionary[key].Substring(0, 4), null, false);
                        subjectFoldersDictionary.Add(newFolder.Id, newFolder.Name);
                        subjectFolder = newFolder.Id;
                        Console.WriteLine("The subject code {0} didn't exist. New folder {1} created in top level folder", subject, newFolder.Id);
                    }
                    if (courseFoldersDictionary.ContainsValue(course))
                    {
                        courseFolder = courseFoldersDictionary.FirstOrDefault(x => x.Value == course).Key;
                        Console.WriteLine("The course code {0} already exists with ID {1}", course, courseFolder);
                        
                    }
                    else
                    {
                        Folder newFolder = sessionMgr.AddFolder(sessionAuthInfo,
                            allRootFoldersDictionary[key].Substring(0, 8), subjectFolder, false);
                        courseFoldersDictionary.Add(newFolder.Id, newFolder.Name);
                        courseFolder = newFolder.Id;
                        Console.WriteLine("The course code {0} didn't exist. New folder {1} created in parent folder {2}", course, newFolder.Id, subjectFolder);
                    }

                    sessionMgr.UpdateFolderParent(sessionAuthInfo, key, courseFolder);
                    Console.WriteLine("Updating {0} into {1}", allRootFoldersDictionary[key], courseFolder);

                    //System.Threading.Thread.Sleep(1000);
                }
            }

            watch2.Stop();
            Console.WriteLine("All folders cycled through in {0} ms", watch2.ElapsedMilliseconds);

            // STOP WRITING CODE HERE

            Console.WriteLine("Press Enter to exit");
            Console.ReadLine();
        }

        /// 
        /// Ensures that our custom certificate validation has been applied
        /// 
        public static void EnsureCertificateValidation()
        {
            if (!hasBeenInitialized)
            {
                ServicePointManager.ServerCertificateValidationCallback +=
                    new System.Net.Security.RemoteCertificateValidationCallback(CustomCertificateValidation);
                hasBeenInitialized = true;
            }
        }

        /// 
        /// Ensures that server certificate is authenticated
        /// 
        private static bool CustomCertificateValidation(object sender,
            X509Certificate cert, X509Chain chain, System.Net.Security.SslPolicyErrors error)
        {
            return true;
        }

        /// 
        /// Creates an auth code. Used when we want to authenticate a user, but don't know their password.
        /// 
        ///The instance name as set in Panopto > System > Identity Providors
        ///Username as defined by Panopto
        ///The full server name as defined by Panopto > System > Settings > General site settings > Web server FQDN
        ///The key produced through Panopto > System > Identity Providors
        /// 
        private static string CreateAuthCode(string identityProviderInstanceName, string username, string serverFqdn, string applicationKey)
        {
            string payload = identityProviderInstanceName + @"\" + username + "@" + serverFqdn.ToLower() + "|" + applicationKey.ToLower();

            var data = Encoding.ASCII.GetBytes(payload);
            var hashData = new System.Security.Cryptography.SHA1Managed().ComputeHash(data);

            var hash = string.Empty;

            foreach (var b in hashData)
                hash += b.ToString("X2");

            return hash;
        }
    }
}

Return to the Panopto API index page

One Comment

  1. Robynne Blissett

    Hi Graham,

    I have found this post very useful, so thank you! Do you have any advice on how I could go about creating a set of pre-defined folders from a list and batch create them under a parent folder?

    1 year ago

Leave a Reply

Your email address will not be published. Required fields are marked *