Life & Work Improvement

Installing Laravel 5 on Ubuntu 15.04

On a fresh install of Ubuntu 15.04, and after doing a
sudo apt-get update && sudo apt-get upgrade

Install VirtualBox and Vagrant:
sudo apt-get install virtualbox
sudo apt-get install vagrant

Install the homestead box:
vagrant box add laravel/homestead
(option 1, virtualbox)

Install git:
sudo apt-get install git

Get Homestead:
git clone https://github.com/laravel/homestead.git Homestead

Go into the Homestead directory and create the Homestead configuration file:
cd Homestead
bash init.sh

Go to this directory
cd ~/.homestead
Where you find the homestead.yaml configuration file.

Create a key
ssh-keygen -t rsa -C "you@homestead"
enter 3 times (keep default location)

Change the key location in the homestead.yaml file in the authorize section (/home/xxx/.ssh/id_rsa.pub) + the keys section (/home/xxx/.ssh/id_rsa)

Create Code folder in your home folder /home/xxx:
cd ~
mkdir Code

Go into the Homestead directory in your home folder:
cd ~/Homestead

Run
vagrant up

Change hosts so you can surf to your Homestead box:
sudo nano /etc/hosts

Add the line
192.168.10.10 homestead.app

Go into the vagrant box
vagrant ssh
cd into the Code directory
cd Code

And download laravel in the default folder:
composer create-project laravel/laravel Laravel --prefer-dist

Now you can surf to http://homestead.app and it should show you 'Laravel 5'.

Moving from MediaWiki to SharePoint O365 - part 6

Now that we have our scripts in place to emulate MediaWiki functionalities, copied over the images and files, and prepared html-pages for all our relevant articles, we can send the pages to SharePoint.

With this script, you prepare the articles one by one, create files for it, and call a PowerShell script which does the actual uploading (see below). Don't forget to update the rootPath to whatever you're using.

<?php
ini_set('display_errors', 1);
$rootPath = "C:/workspace/PortableSoft/UniServerZ/www/wikimigration/articles/";

//get files from 
$files = scandir($rootPath.'articlesToSend');
$i = 0;
$filename = "";

//check if there are files to process
if(sizeof($files)>2){
 $filename = $files[$i];
 
 //skip the "this" and "one up" files
 while(($filename == "." or $filename == "..") and $i < sizeof($files)){
  $i++;
  $filename = $files[$i];
 }
 
 //get the contents of the file, split out title and content
 $page = file_get_contents($rootPath."articlesToSend/$filename");
 //$title = substr($page,strlen("<h1>"),strpos($page,"</h1>")-strlen("<h1>"));
 //$content = substr($page, strlen($title)+strlen("<h1></h1>"));
 $title = substr($filename,0,strlen($filename)-5);
 $content = $page;
 
 //write data to files
 if(file_put_contents($rootPath."powershell/pageName.txt", $title) === false)
  echo "issue writing title to file<br>";
 if(file_put_contents($rootPath."powershell/pageContent.txt", $content) === false)
  echo "issue writing contents to file<br>";
 
 //call the powershell script to upload the article to SharePoint
 $command = 'powershell "'.$rootPath.'powershell/JV.ps1"';
 $output = shell_exec($command);
 //show the output produced by powershell
 echo "<div><pre>$output</pre></div>";
 
 //move the file from articlesToSend to articlesSent
 if(rename($rootPath."articlesToSend/$filename",$rootPath."articlesSent/$filename") === false){
  //if unsuccessfull: give feedback
  echo "<div>issue moving $filename to articlesSent</div>";
 }
 else{
  //if successfull: continue with next file
  echo '<meta http-equiv="refresh" content="3;URL=http://localhost:8080/wikimigration/articles/uploadToSharepoint.php" />';
  echo "<div>file $filename sent and moved to 'articlesSent'</div>";
 }
}
else {
 echo '<meta http-equiv="refresh" content="3;URL=http://localhost:8080/wikimigration/articles/uploadToSharepoint.php" />';
 echo "no files present";
}
?>

Now for the PowerShell script. Put it in a subdirectory powershell in the articles directory. Don't forget to change the siteURL, your username and your password on line 78. And of course the script directory on line 4.
You'll notice the script is calling a dll. You actually need a bunch of them. Create a subdirectory dll in your powershell folder, and put these dlls there: client.dll, microsoft.office.client.policy.dll, microsoft.office.sharepoint.tools.dll, microsoft.sharepoint.client.dll, microsoft.sharepoint.client.publishing.dll, microsoft.sharepoint.client.publishing.silverlight.dll, microsoft.sharepoint.client.runtime.dll, microsoft.sharepoint.client.silverlight.dll, microsoft.sharepoint.client.silverlight.runtime.dll, microsoft.sharepoint.client.taxonomy.dll, microsoft.sharepoint.client.taxonomy.silverlight.dll, microsoft.sharepoint.client.userprofiles.dll, microsoft.sharepoint.client.userprofiles.phone.dll, microsoft.sharepoint.client.userprofiles.silverlight.dll, microsoft.sharepoint.dll. I'm not sure if they're all needed, but hey, this bunch made it work for me. I'll upload a zip of them somewhere later, if somebody needs them and can't find them anywhere else.

# global vars
$clientContext
$rootSiteUrl
$scriptdir = "C:\workspace\PortableSoft\UniServerZ\www\wikimigration\articles\powershell"

function Initialize-SPPS
{
 [CmdletBinding()]
 param
 (
     [Parameter(Mandatory=$true, Position=1)]
     [string]$siteURL,
  
  [Parameter(Mandatory=$false, Position=2)]
  [bool]$online,

     [Parameter(Mandatory=$false, Position=3)]
     [string]$username,

     [Parameter(Mandatory=$false, Position=4)]
     [string]$password
 )
 Write-Host "Loading the CSOM library" -foregroundcolor black -backgroundcolor yellow
 [Reflection.Assembly]::LoadFrom("$scriptdir\dll\Microsoft.SharePoint.Client.dll")
 Write-Host "Succesfully loaded the CSOM library" -foregroundcolor black -backgroundcolor green

 Write-Host "Create client context for site $siteUrl" -foregroundcolor black -backgroundcolor yellow
 $context = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)
 
 $context.RequestTimeOut = 1000 * 60 * 10;

 if ($online)
 {
  Write-Host "Setting SharePoint Online credentials" -foregroundcolor black -backgroundcolor yellow
  
  $context.AuthenticationMode = [Microsoft.SharePoint.Client.ClientAuthenticationMode]::Default
  $securePassword = ConvertTo-SecureString $password -AsPlainText -Force

  $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword)
  $context.Credentials = $credentials
 }

 Write-Host "Check connection" -foregroundcolor black -backgroundcolor yellow
 $web = $context.Web
 $site = $context.Site
 $context.Load($web)
 $context.Load($site)
 $context.ExecuteQuery()
 
 Set-Variable -Name "clientContext" -Value $context -Scope Global
    Set-Variable -Name "rootSiteUrl" -Value $siteURL -Scope Global
 
 Write-Host "Succesfully connected" -foregroundcolor black -backgroundcolor green
}

Function Create-WikiPage([Microsoft.SharePoint.Client.ClientContext]$Context,[string]$WikiLibraryTitle,[string]$PageName,[string]$PageContent)
{
    $wikiLibrary = $Context.Web.Lists.GetByTitle($wikiLibraryTitle)
    $Context.Load($wikiLibrary.RootFolder)
    $Context.ExecuteQuery()

    $wikiPageInfo = New-Object Microsoft.SharePoint.Client.Utilities.WikiPageCreationInformation
    $wikiPageInfo.WikiHtmlContent = $PageContent
    $wikiPageInfo.ServerRelativeUrl = [String]::Format("{0}/{1}", $wikiLibrary.RootFolder.ServerRelativeUrl, $PageName)
    $wikiFile = [Microsoft.SharePoint.Client.Utilities.Utility]::CreateWikiPageInContextWeb($Context, $wikiPageInfo)
    $context.ExecuteQuery()   
}


# get files for variables
    # get file for pagename
    $pageName = [IO.File]::ReadAllText("$scriptdir\pageName.txt")
    # get file for page contents
    $pageContent = [IO.File]::ReadAllText("$scriptdir\pageContent.txt")


# connect
Initialize-SPPS -siteURL "https://xxxxx.sharepoint.com/sites/yyyyy/" -online $true -username "user@domain.com" -password "yourPass" 

# put page
$fullPageName = "$pageName.aspx"
Write-Host "Adding page" -foregroundcolor black -backgroundcolor yellow
Create-WikiPage -Context $clientContext -WikiLibraryTitle "Site Pages" -PageName $fullPageName -PageContent $pageContent
Write-Host "New page added" -foregroundcolor black -backgroundcolor green


$clientContext.Dispose()

Moving from MediaWiki to SharePoint O365 - doRegularPages.php

This script will process the individual html files created from a MediaWiki xml dump by this script to final output files to be uploaded to SharePoint (except for templates, which remains a manual work).

Make sure you've created the folders as explained here, update the URL of your MediaWiki installation on line 5, the URL of your SharePoint site on line 7 and the path of your project directory on line 8.

Moving from MediaWiki to SharePoint O365 - extractPages.php

This is the source code of the extractPages-script, which will create individual html files for all the articles contained in an XML dump from MediaWiki.

Don't forget to update the rootPath on line 2 and the yourSite MediaWiki namespaces on lines 49 - 51.

Moving from MediaWiki to SharePoint O365 - part 5

So we've pimped SharePoint O365 with some javascripts and copied all files and images from our MediaWiki to certain libraries in SharePoint. What remains is getting the textual content there. To do this, we're going to combine the formatted (HTML) content, which we rip from our local MediaWiki website using PHP with cURL, and the unformatted wiki-syntax content which we get from an xml-dump.

Moving from MediaWiki to SharePoint O365 - filePathFlattener.php

This is the source code of the filePathFlattener-script, which will move all files from the images folder-structure of a MediaWiki installation to two folders: one for all the images, and one for all the other files.

It requires there to be an images folder, a filesFlat folder and an imagesFlat folder. You of course also need to update the rootPath on line 4.

Moving from MediaWiki to SharePoint O365 - part 4

Having our scripts in place to emulate some of the MediaWiki functionality and knowing how to use it, we can now finally turn our attention to the actual migration of content. Content comes in two forms: text, as inputted in MediaWiki, and attachments. The attachments can be documents (docx, xslx, pdf, ...) which are available for download, or they can be images (png, jpg, gif, ...) which are displayed in the articles themselves.

In this post, let's focus our attention on the attachments.

Moving from MediaWiki to SharePoint O365 - part 3

Now that you have the scripts in place, how do you make use of them?

Redirect

This one is simple. Write #REDIRECT[[pagename]] like you're used to doing it in MediaWiki. Just take care that you write it exactly as I did: in uppercase, without a space between # and REDIRECT or between REDIRECT and [[. My script is not dummy-proofed like MediaWiki is.

Math and source code

As we're making use of third party scripts (MathJax and SyntaxHighlighter), you should refer to the corresponding websites for documentation. In short, you can do the following:

Moving from MediaWiki to SharePoint O365 - part 2

After getting our tools set up and acquainting ourselves with SharePoint O365, lets take a step back and think for a moment. The objective is to migrate all wiki content from MediaWiki to SharePoint O365. We can get to our content in MediaWiki, i.e. the raw content as the user types it in, or if need be the HTML-content produced by MediaWiki (using PHP with cURL). And I've got a way to upload HTML-content into pages in SharePoint. So this doesn't sound like such a challenge after all?

Wrong.

MediaWiki has a lot of user friendly formatting syntax. Like * for bulleted and # for numbered lists. Like ==heading==. Like [[some other article]]. Or how about [some external link]. And don't forget the [[Image:something|thumb|right]]. And then there's categories. And templates. And wikitables. And if you've had a bit of fun with the extensions, there's code highlighting. And math. And lots of other stuff.
SharePoint has HTML and an editor.

Moving from MediaWiki to SharePoint O365 - templateCaller.js

This is the source code of the templateCaller script, which, as mentioned, does more than replacing templates by now.

To work, it requires to be placed in your SiteAssets/JayVee folder (create the JayVee folder), where there should also be jquery.min.js and the folders MathJax and SyntaxHighlighter with the corresponding scripts inside. Your seattle.master should be updated to load the jquery script (first) and this script (second) as explained here.
You of course need to change the vars sharePointRootURL (line 15) and sharePointSiteRootURL (line 16) to match the URLs of your site.

If there are errors in it, or things that can be done better, please let me know in the comments so I can improve it! I am not at all fluent in javascript / jQuery, nor will I pretend to be.

Moving from MediaWiki to SharePoint O365 - part 1

Some clever person at Microsoft understood the wiki hype and decided to call some component of their SharePoint platform 'wiki', never mind if it really is one or has some of the much-used functionalities of such things. They knew management would fall for it. And in my case (and yours apparently), they did. It's just another wiki right?

From the articles I've written before, you know I'm a fan of good software being used for what it's meant to do. Like MediaWiki for wikis. So if you've ended up on this page because you're being forced to move away from MediaWiki instead of towards it, let me tell you: I feel your pain. I understand your frustration. If you've never worked with SharePoint before, I understand the frustration you're about to have.
But let's not mix these feelings with the work that needs to be done. So here's some empty space where you're allowed to feel sorry for yourself:

... and done.

Now we get to work.

CoolTools - SphinxSearch

In the previous posts, we've set up a virtual machine, installed ubuntu, got the prerequisite server software ready and a basic install of MediaWiki running and reachable as your intranet wiki.
But we're not quite ready to go live. In this and the next post, we'll be installing Sphinx as your back-end search engine and doing some additional configuration work.

People will be continuously searching through your wiki for that exact bit of information they need, so it follows you'll be needing a good search functionality. The basic search included in your wiki installation is not a good search functionality.
SphinxSearch is a (stand-alone) lightning-fast indexing and searching engine. We'll install it, then give it access to the MediaWiki MySQL-database . Finally, we'll install the sphinxsearch-mediawiki-extension to feed search queries from wiki users to Sphinx and results from Sphinx to your wiki.

I realize this post is a bit extensive, but rest assured, it's worth it.

CoolTools - MediaWiki

After the installation of the free web server and other required software on the free operating system Ubuntu, which we've set up in a Virtual Machine run by the previously installed - and again, free - VirtualBox, we're now ready to install our own wiki.

The software we'll be using for that is MediaWiki:

MediaWiki is a free software open source wiki package written in PHP, originally for use on Wikipedia. It is now also used by several other projects of the non-profit Wikimedia Foundation and by many other wikis.

CoolTools - Setting up the web server

In the previous articles, we've installed our virtual machine player and set up a virtual machine with Ubuntu.
Now it's time to make our machine a real web server. The actual “web server” software is Apache HTTP server, but we’ll also be needing PHP for dynamic websites like the wiki, MySQL as the database of choice, and PHPMyAdmin as a GUI on MySQL. These are typical tools for web servers.

Start up your virtual machine, and start up a terminal window. You can do this via Dash, the uppermost button in the shortcut bar on the left of the screen of your virtual machine. In the search box, enter “terminal”. This will open a console within the GUI, like cmd would on a Windows machine. We’ll be using this terminal a lot.

CoolTools - Ubuntu Operating System

Open source tools and websites tend to get better support on free operating systems. For this series, I've chosen to go with Ubuntu. No particular reason.

There are specific editions of Ubuntu for desktop, server, mobile, tablet, TV... You'd think I'd go for the server edition, but I won't. I'm not enough of a guru to do without some GUI backup :-)

Head over to the Ubuntu website and download the Desktop version. I've gone with 12.04 because that is an LTS, or Long-Time-Support release. Remember where you stored it, we'll need it in a minute.

CoolTools - Virtual Machine

When you're experimenting with new software, you don't want to endanger your existing set-up. The ideal way to go is using a Virtual Machine: you create a virtual computer inside of your existing computer and install everything there.
The advantages are many:

Only 1 thing to install on your existing set-up: the Virtual Machine player.
If you mess something up: just throw away your Virutal Machine and start again.
If you end up with a smoothly running Virtual Machine, and you want to experiment some more without endangering what you've already done, you can easily clone your existing VM and experiment on the clone. If you're happy with it, start using the clone. If not, stick to the original.
If you work in a big company, you'll probably be trying this out in your team/department. If you & your management get convinced of the added value, you can turn over the Virtual Machine to your IT-guys: there's a good chance your company is already using virtualized servers, which means they can most likely just drop the VM in their cloud and be up and running in 5 minutes.
You can use a different operating system than the one your computer is equipped with. I.e. if you have an MS Windows computer, you can install the Ubuntu operating system in the Virtual Machine inside of MS Windows. Why not use Windows? To use the built-in features of Windows as a web server, you'd need extra licences as soon as the number of connections gets to be too high (or just buy their server software: Windows Server). We want this to be free. Also: the software we'll be using tends to get better support on non-Windows operating systems.

There's actually only one disadvantage I can think of, and that is that your computer needs to run 2 operating systems instead of one, which means a lower performance. However, this disadvantage is a rather theoretical one: you don't need much power for running the tools I'll be describing, and as long as your public is limited to no more than a few hundred users, you'll be fine. And if you find you're serving more than a few hundred users (at which time you can definitely call your experiment a success!), you can throw the VM in a cloud to give it more power.

To work!

Installing the VM-player

The Virtual Machine player of choice is called VirtualBox. It's an Oracle product (the company known for their Oracle database). Although Oracle is a commercial enterprise, VirtualBox is free. In a following article, we'll be installing MySQL, a free database system also available from these guys.

Ok, to get started. Head over to http://www.virtualbox.org and download the appropriate software from their Downloads page. You only need the VirtualBox platform package, the extension pack is not free for commercial use. If you have a Windows computer, download the one described as VirtualBox ... for Windows hosts.

Once downloaded, install it. The installer makes this a pretty straight-forward process, you shouldn't have any issues here.

Now that we've got the VM player in place, it's time to install ourselves an operating system. I've chosen Ubuntu, and will describe the installation in the following article.

CoolTools series

In this series of articles, I will provide a walk-through on how to enrich your work environment with some shiny 21st century software. Whether you're a small or even a bigger enterprise, stuff like this will enable your employees/coworkers to perform better, smarter, more efficiently.

And, because CoolTools is aimed at companies, all the software mentioned is free for commercial use. That's right: it literally won't cost you a penny in licenses. All you need is a computer plugged into your network: an old spare one if you have it, or if you don't have one to spare, a computer in use will work as well (provided it's not too old).
Apprehensive? Not wanting to mess up settings and doing a lot of installation work on one of your existing, working computers? No worries. We'll install everything we need inside of a virtual machine, so there's only one thing you need to install on your existing computer: the virtual machine player. Everything else is inside of the virtual machine, and if something gets messed up, you just delete it and start over. Hassle free!

Ready to get started? Check the rest of the series!

Getting started with the Virtual Machine -->

Redirect

Math and source code

Installing the VM-player

Next