MIDAS#12 Out of Notes Part 5 : To SharePoint (Midas Rich Text LSX)
Out Of Notes, part 5: To SharePoint shows how a Notes database can be exported to CSV (comma-separated-values), the technique we have developed with a few early adopter companies to take Notes data to SharePoint with maximum flexibility and fidelity. In this export, all fields can be included or only a subset. Rich text fields are converted to HTML or XHTML while images and attachments are extracted as separate files which can be loaded to SharePoint in a separate sweep. Since a CSV file is a plain text field, it is also available for easy post-processing in Excel or using various tools that work with CSV files.
The export to CSV can work with any size collection, and a size limit can be established so that a series of CSV files can be created for very, very large databases. The export is very fast. In this demo, a single CSV file is created for the 124,200 documents, each of which has four rich text fields with total of over 3.5GB of extracted images, and the entire export takes approximately 20 minutes.
The agent used for the export is shown below. This agent will only work in Midas V5.00, and exports of over 1000 documents require the Midas Exports add-on license. An additional feature is that this does not require access to the design of the database, but can extract the data from locked-design databases as well.
This extended sample requires Midas 5.00 or higher. If you would like to try out this sample, request an evaluation license, and specify that you want to try Version 5, and one will be mailed to you as soon as the new version is released.
AGENT: Export to CSV
Set coll = db.Search(|Form="|+form.name+|"|, Nothing, 0)
If coll.count > 0 Then
Set gcoll = New GeniiCollection
Set doc = coll.GetFirstDocument
count = 0
While Not (doc Is Nothing)
If gcoll.AddDocBackend(doc.Handle) Then
count = count + 1
If count Mod 1000 = 0 Then
Print CStr(count)+" documents added for Form '"+form.name+"'"
Set doc = coll.GetNextDocument(doc)
Set doc = Nothing
Print "exportsubdir-"+form.name+": "+CStr(count)+" of "+CStr(coll.Count)+" documents collected. Exporting..."
successful = gcoll.ExportToCSV(exportdir+"\"+exportsubdir+"-"+form.name+".csv", actiondoc.Properties(0))
Rough transcript (used for Closed Captions as well):
This is Ben Langhinrichs of Genii Software with a mini-lesson called Out of Notes Part 5: To SharePoint. In this case, we're going to take a version of our Curb Appeal daatabase that I've been showing you which has house listings. This one is a much larger version.
It has 124,200 documents and has about 3.6GB of information. We're going to export it to a CSV (comma-separated values) in order to be able to import it into SharePoint. If we look in this, we'll see that each field has a photograph, and it has four different rich text fields and a number of other values.
So now what we're going to do is export this to our CSV file, and we use this helper database we have. First let's export the Curb Appeal database which just has 46 documents, and that's practically instantaneous, but if we go look, it will give us an idea of what we're getting.
We have a CSV file right here and we just open it in Excel. You can see how each of the fields is represented, and how there is HTML for the rich text fields. So, if we go back out, we'll see that there are references to photographs, and one that has the photographs. Those will then be extracted and can be loaded into SharePoint separately.
After, we go in here and here are all those photographs. But let's take a look now at the much larger database, and let's start the export.
In reality, this takes about 20 minutes, but I sped it up for the purposes of this demo so you can see roughly what it does. It's going to go and extract all of those images, the 3.6GB of images, and the CSV file and it's going to take it out to those same directories where it can be imported.
See, right there it has exported everything. Now if I go back and look at the directory, you'll see that there are a lot of images. In this case, there are 124,200 because there's actually exactly one for each document. If we look at the CSV file... I'm not going to try to open it right now, but it's 800MB. That's big, but it'ss easy to load into SharePoint. I've done this with customers with much larger ones than that actually.
And that's what it takes to take your data out of Notes, an entire database, it can even be a locked design, take it out and export it to SharePoint.
Thank you very much and stay tuned for more episodes.