• There will be no Performance Point, Reporting Services or PowerPivot in SharePoint Online. Excel, Visio and MS Access Services will be there but they can only connect to a data source via Business Connectivity Services. PowerPivot is targeted for release in FY12.

• Existing Enterprise Agreements require a user to have a license for each device on which they want to use MS Office. Office Professional Plus (Office 365 offering) will allow a user to use the same license on up to 5 devices.

• It is possible to synchronise your local Active Directory with the Office 365 one.

• Office Web Apps will be utilised if the Office Professional Plus offering is not taken.

• SQL 2012 features such as Power View, will only be considered a year after the official release of SQL 2012.

• Windows Intune and CRM Online are not yet ready for the South African market.

• The primary Microsoft Data Centre is hosted in Dublin, while the failover one is hosted in Amsterdam.

• Due to all the cloud service offerings, one will need to consider the data cost when looking at Office 365. One may need to include the customer’s ISP during the discussions.

• Some organisations may not want to store their data in another country. Fact is there is no law prohibiting South Africa organisations from storing their data elsewhere.

• Project Server is not supported in Office 365.

• An Office 365 Trial will be available in the next few weeks. One will be able to buy licenses within the first 3 months of 2012.

• Standard and Enterprise versions of Exchange and SharePoint are not available in Office 365. The Office 365 Plans will determine which services are available in the specific product.

• There are no out-of-the-box capabilities to migrate on-premise SharePoint sites to Office 365. Some ISVs do offer this service by wrapping the List Service.

• The maximum size of a SharePoint Online implementation is 5TB. There is currently support for up to 20k users, but this will be increased to 100k next year.

Enriching Your BI Semantic Models with DAX

- Some nifty DAX functions: DISTINCTCOUNT, RANKX, TOPN, SWITCH (use instead of nested IIF statements), HASONEVALUE (returns true or false), USERELATIONSHIP (use an inactive relationship instead of the default active one), PATH (use for parent-child relationships), PATHITEM (gets a specific item from the defined PATH), LOOKUPVALUE (get an attribute based on another value), USERNAME (can be used for row level security)
- Week time intelligence functions are not available. One will need to add in the relevant attributes to your calendar dimension and then write a custom calculated measure
- There will be a DAX query plan! Available as a new option in SQL Server Profiler

SQLCAT: Configuring and Securing Complex BI Applications in a SharePoint 2010 Environment with SQL Server 2012

- When setting up your SPNs, use both the fully qualified and net bios user names
- Use the -S SPN option: will ensure that duplicate SPNs are not added. Use -X to list duplicate SPNs
- Important to create a service account that will be running the Claims to Windows Token Service. This service will be on all servers that have SharePoint installed
- Constrained delegation will not cross domain boundaries, no matter what
- On the Delegation tab, ensure that the Use any authentication protocol option is selected
- Kerberos does a lot of caching. Some of the caches can only be cleared after rebooting the servers
- Kerberos only needs to be configured for PowerPivot if one of the data sources is a linked SQL server
- Power View does not support the prompting of credentials on an .rsds connection file

Using ColumnStore / Vertipaq Indexes in SQL Server 2012

- Huge, huge, huge performance gains!
- ColumnStore indexes will not always be the best solution e.g. for very restrictive queries
- All data for a single column is stored on a separate page; access only the columns you need
- At some stage the BISM model will struggle with huge data volumes. Because of things like Direct Query, one can use the ColumnStore index to increase performance for BISM models
- Some really smart stuff has been done from an encoding point of view. E.g. Instead of storing the true sales number, an offset is stored and all numbers are divided by this offset resulting in smaller numbers that need to be stored
- Run Length Encoding is also another smart way compression is being handled
- Data can be used in query processing without decompression
- On average, compression will be 1.8 times more effective than page compression
- Consider using ColumnStore indexes on tables that have 5 million + rows
- Create the index on ALL the columns. You can only have one ColumnStore index per table
- Outer joins and Unions will mess with batch processing

Wayne Snyder was an outstanding presenter, loved his ‘old-man with 100 years experience’ presenting style.

And that concludes a great SQL PASS conference! There is some game-changing content on the way and I can’t wait for SQL Server 2012. Next year’s conference will be held in Seattle once again in early November.

Thursday, Opening Keynote

- The Fantastic 12 of SQL Server 2012:
+ Required 9′s and Protection
+ Blazing Fast Performance: Performance enhancements to the RDBMs, SSAS and SSIS. And then there is the new ColumnStore Index – really looking forward to using this one in our DW environments!
+ Rapid Data Exploration: Power View and PowerPivot 2.0. Great features and enhancements in this space. Also included is the new Reporting Alerts capability – ability to send out alerts based on data conditions
+ Managed Self-service BI through SharePoint
+ Credible, Consistent Data: BI Semantic Model, DQS and MDS
+ Organisational Compliance: Expanded Audit – User-defined, Filtering, User-defined Server Roles
+ Peace of Mind: Production-simulated Application Testing, System Center Advisor and Management Packs, Expanded Support – Premier Mission Critical
+ Scalable Data Warehousing
+ Fast Time to Solution: SQL Server Appliances – optimised and pre-tuned
+ Extend Any Data, Anywhere: ODBC drivers for Linux and Change Data Capture for SSIS and Oracle, Semantic Search
+ Optimised Productivity: SQL Server Data Tools, Unified Across Database and BI, Deployment and Targeting Freedom
+ Scale on Demand: AlwaysOn, Deployment across public and private, Elastic Scale (scaling in the cloud with SQL Azure Federations)

Data Quality Services – A Closer Look

- Setup your Knowledge Base and then perform a Knowledge Discovery exercise. This is similar to profiling your data and will populate the Knowledge Base
- Spelling mistakes are highlighted just like in Microsoft Word – very nice
- Next up, create a Data Quality Project. This will enable you to validate your data according to the rules that have been defined in the Knowledge Base. Corrections will be made and alternatives suggested
- Data that needs to be cleaned can also be validated against external data sources. Only Azure Data Market right now
- One can setup rules to override the corrections provided by the external data provider
- Term-Based Relations: find and replace
- Matching: one has the ability to create a composite domain (using a delimiter to split out multiple fields)
- New feature – Use Knowledge Base Parsing: use this along with the delimiter to use your Knowledge Base to automatically fix mistakes. E.g. If users had swopped around fields in certain instances causing the parsing to be invalid
- Create a Matching Policy and define weightings to the fields you want to match on
- Create a Data Quality Matching project to determine the duplicates in your data and you’re all done!
- The matching algorithm is a black box, not too much detail given here

DQS looks like quite an ok product, considering it is only the first release. Only downside is the fact that South Africa will not be able to access most of the external data sources and the ones they can access may not be that useful.

Developing and Managing a BISM in Analysis Services

- Microsoft continuum: Personal BI (PowerPivot), Team BI (PowerPivot for SharePoint) and Corporate BI (Analysis Services)
- Row and Dynamic security will be available!
- SQL Server Data Tools is the new name for BIDS
- Add calculated measures in the measure window. [Measure Name]:= [Measure Formula]
- Security: read permissions will give you access to the entire database. Use a DAX filter to further restrict the data
- The Tabular model, which contains a GUID in the name, talks to the Workspace Database – a place where all development changes take place
- When creating a new Tabular project, use the Preview and Filter option to restrict the number of rows that are imported into the model. This is particularly useful when dealing with large datasets. A partition will automatically be created for this subset of data. Once the data has been imported, create additional partitions for the rest of the data. The Process All option will bring in the new data
- One can copy data from Excel and paste it in as a new table in Visual Studio. Very nice for prototyping
- One can hide a Security table by applying a DAX filter of =FALSE(). Hiding the table from the client tools does not hide it altogether
- DAX security filter example …security[allowed region], security[login], username(),…
- One can process a database, a table or a partition
- The Process Data option only processes data, calculated columns will not be updated. One will need to Process Recalc afterwards before you can Evaluate (return rows from) the table
- Use Process Add to incrementally load data into new partitions. Use Process Defrag afterwards
- Automatic partitions can be added using AMO
- Use the Extension Manager in Visual Studio to include the DAX Editor – to be made available on CodePlex soon! Great add-in for formatting DAX
- There is also another sweet add-in to script out all the DAX measures into a separate window, all with auto-complete. Cheers MDX!

Cathy Dumas did an excellent job with this session, I’ll definitely go and see her in the future again.

Building the Perfect BI Semantic Model for Power View

- Power View allows for the creating of different views in the same report, the same way you would add a new slide to a PowerPoint deck – very cool feature
- Power View reports can also be created from .rsds connections
- One can perform data mining in BISM by creating a connection to a SSAS source which contains the Mining Model
- In SharePoint, on the .rsds connection properties, check the Set execution context property if Kerberos is not setup. The logged in user’s credentials will then be passed through to the data source
- For numeric attributes, like year, set the aggregation property to Do not summarise. This attribute will then not be aggregated when creating a pivot table report
- Development tip: create a BISM connection to the Workspace database instead of the one that is deployed – speeds up trouble shooting etc.
- Descriptions defined against tables and measures will be shown as hover text in Power View
- Set additional Default Label and Default Image properties on attributes in SQL Server Data Tools. Power View will then know which fields to use when switching between card and chart views

The Power View demo by Carl Rabeler was not great as he had to look down at his script the whole time. Kasper de Jonge was a lot better and gave good insight into the technical bits of BISM.

I can’t wait to start using this technology – hopefully another CTP release will be made available early in the new year…

Opening Keynote

- SQL Denali is officially renamed SQL 2012 and will be released during the first half of next year. I.e. 30 June 2012
- Project Crescent will be known as Power View. It will be available on most mobile devices and is loaded with quite a few new features compared to CTP3. Further down the line (post the SQL 2012 release) one will be able to take a pic of a movie (for example) and use the Bing imaging service to identify the movie cover and filter the report for that movie!

Amir Netz did an excellent job showing off Power View using a movie sales dataset. I’d love to get hold of it for client demos. My only concern is that Power View is still going to be tightly coupled to SharePoint Enterprise (aka the centre of the Microsoft universe). Small to medium size clients will then not be able to leverage this great reporting tool.

Visual Report Design: Bringing Sexy Back

- Principles of effective design include: form (visual design skills), fit (functional business skills) and function (technical skills)
- See http://www.colourlovers.com/ for palette colours and examples of which colours work well together
- Items to consider for your report design:
+ Can the user glean the information in 5 seconds or less?
+ Know your audience
+ Get a second opinion
+ Maintain balance
+ Use POCs and throw them away
+ Accept feedback and be willing to learn

Remember, the novelty of sexy graphs and gauges will disappear over time. Users will just want to get the relevant information to make the decision.

What does it take to add MDS to your DW?

- Create your MDS model in the web UI and then the entities through the new MDS Excel add-in
- Use Attribute Properties to create a relationship between entities. E.g. Product and Category
- Constrained lists can also easily be created using the data in existing columns. I.e. Creating entities for lookups
- Create derived hierarchies back in the MDS UI. Users can then change the hierarchy setup by dragging and dropping items in the tree view
- Add in any required business rules and validate the data through Excel or the MDS UI
- Annotations can also be added to changed data before it is published back to MDS
- Subscription Views make available the data to other systems that need to consume it. Parent-child hierarchies can also be flattened out using the views
- One can install MDS 2012 and still use SQL 2008 R2 on the database engine side, you don’t need to upgrade the entire SQL environment
- Microsoft believe MDS can scale into the millions of records to easily manage customer master data, for example. I’d like to test this out

Introduction to Data Mining in SQL Server Analysis Services

- Great introduction by Brian Knight: stating all his credentials but then down playing them at the same time using a split screen
- Data mining skills will get you a salary increase (maybe)
- Clustering is great for fraud detection. It will measure the distance between two points on a scatter chart and determine a likelihood
- See http://www.sqlserverdatamining.com
- The key is to flatten out your dataset. However, when creating a Mining Structure you can use nested tables if there is a relationship defined
- SAS has good data mining algorithms, Microsoft’s ones are about an 80% fit. The clincher, you already have the MS ones and no additional purchase is required

I’m going to give this a whirl on some client data when I’m back at the office. Data mining, although not that well used, can add a lot of insight to the existing data warehouse.

Brian was the best presenter of the day for me.

On the brighter side were the friendly people, amazing buildings and malls you could never even dream of – full with ski slopes, aquariums and 1,200 stores!

Ok, so why were we out there? A potential client had asked us to review their infrastructure, business processes and technical implementation with the core focus relating to operational efficiencies. I was there more from a technical perspective and had the exciting job of building out a few POCs while we were on-site. This is where I have to give credit to the Microsoft stack, especially to PowerPivot. In just a few hours one is able to mash up multiple data sources and present back analysis that the client thought would take a very long time to complete. Our rapid BI development tool, timeXtender, also came in very handy when building out a finance cube on a subset of the client’s database.

These tools not only accelerate the ability to quickly create POCs, but also significantly reduce the development time for full-scale projects. The company I work for has recently had numerous success stories where ROI has been realised very quickly, sometimes within the first month of going live! The common denominator – timeXtender.

The review came to an end all too quickly but I was happy to be back in SA again. There is nothing quite like your own bed. Well, we’ve compiled an exciting roadmap for the client – let’s hope they want us to partner with them on the journey.

The past month has seen me building a virtual machine containing all the new features in the next version of SQL Server, specifically the BI ones. Top of my list was Project Crescent along with the new version of PowerPivot. I’ve also been delving into the new version of Master Data Services and started taking a look at Data Quality Services today. Here is my impression on what I’ve seen so far:

PowerPivot

The new release of PowerPivot is really great and I am impressed with a lot of the new capabilities including: KPIs, hierarchies, diagram view, multiple relationships on the same field, meta data, advanced column sorting, perspectives, measure formatting and the ability to show details by right-clicking on the PivotTable. I also like the fact that you can change a field property to ImageURL (Advanced Properties) and then have these images appear in your Crescent reports.

To get a full list of the new features, open the PowerPivot window, click on the Help icon and then select the What’s New menu item.

Project Crescent

Wow, I really like Crescent! Crescent reports are pretty much Silverlight PivotTables based on PowerPivot models (uploaded to SharePoint 2010) or the new Tabular version of Analysis Services. The reports are very dynamic and filtering in one area automatically filters data in the other PivotTables and graphs (called Highlighting). The scatter chart also has an option for a Play Axis which is great for comparing data over time. Some of you may remember this report in the PowerPivot Management Dashboard in SharePoint’s Central Administration. Below is a screen grab of the report I created from my rugby PowerPivot data.

The only downside I see to Crescent is the fact that you need SharePoint 2010 Enterprise Edition in order to get access to any of the cool BI features. This is going to prevent a lot of customers leveraging the new technology. And currently, I’m unaware of any applications which will be able to render these reports on a mobile device.

Microsoft Visual Studio 2010

So far I’ve explored the Tabular version of Analysis Services. It’s PowerPivot in Visual Studio (in blue instead of green) with a whole bunch of over advanced features e.g. Partitions and Roles. All up it looks decent. When embarking on a new project, architects will now need to decide as to whether they go the classic Analysis Services route (as we know it) or if they go the Tabular route.

Only thing I couldn’t see was how to add calculated DAX measures in my Tabular project…

Master Data Services

I must say that I was looking forward to an upgraded version of MDS but was disappointed to see that the majority of the application was unchanged. There are some Silverlight screens now embedded in the UI to improve the experience and performance but that was about it. There is also an Excel 2010 MDS add-in that allows you to connect to an MDS instance. This add-in will enable you to manage your master data in a very familiar environment to all of us. I’ll need to explore this feature in more detail to see how robust it is.

Data Quality Services

As I mentioned earlier, I only started looking at this new addition earlier today and my first impressions are good. I setup a Knowledge Base with some Domain Rules and Term-based Relations and it worked well. The UI also looks decent – a lot better than MDS. Below is another screen grab.

What will be interesting to validate is the matching capabilities as one really wants to be able to easily de-duplicate datasets.

Stay tuned for more…

Instead of using the Contoso or Adventure works datasets to build out my dashboard, I went for a dataset I came across last year which revolves around Tri-nations rugby players. Quite interesting (if you like rugby that is). I used this same dataset when I showed PivotViewer at Tech-Ed Africa 2010 and received some good feedback.

Dashboard Designer allows you to create a host of different objects including KPIs, Filters, Reports, Dashboards, Indicators and Scorecards. Dashboards are the containers and allow you to link up the other object types in one location. Once deployed, a Web Part page will display your dashboard. The only downside to this approach is that if you want to include document libraries and team discussions (as an example) into your dashboard, Dashboard Designer will overwrite those web parts the next time the dashboard is deployed.

Now, you can easily get around this dilemma by not using the dashboard component in Dashboard Designer but instead create a new Web Part page from scratch. One can then quickly include your published Performance Point content in these web parts as well as your document libraries, custom lists or team discussions. One trick, you need to create connections between your different web parts in order for filters to work or for additional reports to be generated once a user clicks on a particular item. Note: connections would also need to be setup in Dashboard Designer.

Here is what my dashboard looks like when you access it for the first time:

Once the user clicks on a player position or KPI, additional reports are displayed:

I’ve even embedded the new Strategy Companion Analyzer Scorecard component:

And finally a little fun with my team discussions. I’ve included some classic one-liners from our coach, Pieter de Villiers.

With just a little bit of luck Oregan Hoskins will see this dashboard and recruit me as a data analyst for the up-and-coming Rugby World Cup!

I’ve been pretty busy lately building POCs so when a new one came by one of our Account Executives decided to help me out by building it himself (Schneeberger likes this). We had a good model to start with and building out the reports would also not be too hectic.

The client feedback session was setup for 09:00 this morning and let’s just say that our model wasn’t working as expected. So at 07:45 I was trying to fix the model while the AE was building out some reports. We left for the client at 08:15 leaving me some time to finish off the reports in the car. Well we hit another road block when the AE’s VM blue-screened on the way there. At 08:40 I started loading the client’s data into the model (using my bosses SUV boot as a desk) on my VM. The data completed loading as we sat down in front of the client. Could this really be happening?

It ends up the client is not too interested in the model but more in the reports that can be created from it. So off I go building out reports in Excel 2010 and the client is loving it. Now that’s what I call Agile BI!

I believe there was a fair amount of luck involved but it also confirmed that the BI model we are using is really solid. Testament to all the hard work that has been put in to accelerate BI implementations.

I called all the launch stores and noticed that the iStore on Sandton Drive was the only one to open at 07:30am. To guarantee myself one of the devices I left home real early and was there at 03:55am. After spending quite a bit of time explaining to the security guards why I was there that early, they finally gave in and opened the gate. I was the first person for an hour before the second chap arrived. Let me also add that it was 3 degrees outside which was not that pleasant.

So, 07:30 arrived and there were a lot of people queuing up (behind me of course ). The Apple staff were really well organised and handed everyone numbers and also allowed you to pick your specific model based on availability. I went for the top of the range 64GB, Wi-Fi & 3G along with one of those magnetic covers. The wait was extended by another hour and a half as national sales could only kick-off at 09:00am. Come on! Anyways, at 09:05am I had the iPad 2 in my hands and the 5 hour stint was over.

I believe the device is miles ahead of any of the other slates in the market right now. I can’t wait to explore the business intelligence capabilities as one of our products, Strategy Companion Analyzer, has just released their beta mobile version.

Here are some pics of the event and device.

Rewind to 2005. I was working on a fairly large project and the business sponsor felt it necessary to appoint an independent project manager to facilitate meetings and take notes. All went fine for the first few months but it became increasingly difficult to do what the PM wanted as it sometimes conflicted with the business sponsor’s view. This was an easy one to sort out for me; contact the business sponsor and get them to confirm that what the PM was doing was not aligned correctly with their vision. Great, got the confirmation and forwarded it to the PM. This is where it all went pear shaped. The business sponsor and PM were obviously good mates and going behind the PM’s back unleashed a beast worse than the one in Beowulf! The mudslinging started and I was called a cockroach and some other interesting names. Fun and games! I printed out the name calling emails and stuck it next to my desk for some motivation.

Then in 2009 I had another ripper with a gentleman at one of our big clients. He went by the title of Managing Director. This project was huge and we had been dealing with the entire executive team since inception. The Initiation Phase went on for the good part of 5 months where after we started some actual projects. One of the first projects involved setting up a Balanced Scorecard followed by some data warehouse foundations. I needed to meet with one of the staff reporting to the MD regarding some data I required and went about it without informing him. Big mistake. Next thing this staff member had forwarded my request to him asking how they should proceed. Give me the friggen data, that’s how! No, that’s not how it went. Mr MD became all aggressive and proceeded to rip me a new one (via email of course) stating exactly what he thought of me and the project. Needless to say he handed over the data the following day after some clarification of course – funny man.

Fast forward to just a few months ago where myself and tag team partner Chris Barry (http://techpov.wordpress.com/) were grafting at a very interesting site. We had met with the Financial Director and Franchise Executive to get some requirements for the Proof of Concept (POC) we had to build. The FD wasn’t too concerned, as long as we had covered the Franchise Executive’s requirements. Off I went to build, what I thought, one of my best POC’s to date. A feedback session was arranged and I was looking forward to blow the socks off these guys. The first issue arose when the Franchise Executive cancelled but asked us to continue with the FD. All good. So I start demoing my reports (which I believed were cooking with gas) only for the FD to stop me 5 minutes in and tell me that he was underwhelmed by what he was seeing. I was speechless. Chris turned and gave me a look to say “unlucky mate, you’ve on your own now!” Right, so I wiped the egg off my face and got the reports that the FD really wanted to see. Long story short, we presented again a few weeks later and the FD was a huge fan of the new reports. Mark: 1 FD: 1

And then, to cap things off, I had another memorable experience with the CIO at a different client. We were still in the initial phases of the engagement and there was a debate regarding how we should approach one of the projects. I piped up and said that we will be delivering, as part of the deliverables, a future architecture that we should be striving for. For some reason that didn’t go down as he turned to me and said “I don’t give a f%#k about the architecture”. Right, no problem mate glad we cleared that up. Anyways, Chris and I presented feedback last Friday to all the executives (excluding the CIO as he was overseas) and they loved it. They want to know how long it will take for the implementation to be completed.

Ok, so those were some of the more interesting times with the executives that I can remember. I’m sure I’ll be adding to this list in the future.