Reading Excel from .Net: Creating a Cell structure

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/reading_excel_from_net_creating_a_cell_structure.htm]

I've been blogging a series on enhancing Excel with .Net. One way to do this is to have some process in Excel kick off a .Net Console app, and than have the .Net app do all the difficult programming (where you have the support of the .Net Framework) as opposed to VBA. In order to do this, we need to be able to read an Excel sheet from .Net.

Previous posts in this series were:

• Migrating an app from Excel to .Net, Part I
• Excel Migration Part II: Calling a .Net Process
• Extending Excel with .Net Part III: Context Menus

This post will start showing how to read Excel from .Net. This is pretty standard. If you google it you'll see a lot of variations. However, many "solutions" are not very reusable. We'd like to create a reusable ExcelReader utility class. This will be much easier if we first build a "Cell" class. This Class should have:

• Constructors to take a row and column in both RC (int-int) and A1 (letter-int)  notation.
• Properties for the row and column (even if you use the A1 notation). So if you enter the column 'C', it will return 3.
• A Property to always return the A1Reference. So if you enter (2,2), it should return "B2".

This post will show how to build and test this cell. The next post will apply it to our ExcelReader.
 

Source Code:

using System; namespace ExcelUtilities {     public struct Cell      {            #region Constructors         public Cell(int intRow, int intCol)         {             if (intRow < 1)                 throw new ArgumentException("Row must be greater than 0");             if (intCol < 1)                 throw new ArgumentException("Column must be greater than 0");             _intRow = intRow;             _intCol = intCol;             _strA1Reference = ConvertIntToStringColumn(intCol) + intRow;         }         public Cell(string strCol, int intRow)         {       if (strCol == null)         throw new System.ArgumentNullException("Column can't be null");       if (intRow < 1)         throw new ArgumentException("Row must be greater than 0");             _intRow = intRow;             _intCol = ConvertStringToIntColumn(strCol);             if (_intCol > 256)                 throw new ArgumentException("Column cannot be greater than IV (256). Value was '" + strCol + "'.");             _strA1Reference = strCol.ToUpper() + intRow.ToString();         }                   #endregion         #region Implementation         private static int ConvertStringToIntColumn(string strCol)         {             strCol = strCol.ToUpper();             //A --> 1, BD --> (26+4), IV                  //Can only have max of two characters.                  char[] ach = strCol.ToCharArray();             if (ach.Length == 0)                 throw new ArgumentException("Column cannot have 0 length. Must be A - IV. Was: '" + strCol + "'.");             else if (ach.Length == 1)                 return GetNumberValue(ach[0]);             else                        return 26*GetNumberValue(ach[0]) + GetNumberValue(ach[1]);         }         private static string ConvertIntToStringColumn(int intCol)         {             if (intCol < 1)                 throw new ArgumentOutOfRangeException("intCol",intCol,"Column must be between 1 and 256.");             if (intCol > 256)                 throw new ArgumentOutOfRangeException("intCol",intCol,"Column must be between 1 and 256.");             //single digit, just convert it.             if (intCol <= 26)                    return GetCharacterValue(intCol).ToString();             //multiple char             int intDigit1 = intCol / 26;             int intDigit2 = intCol % 26;             string s = GetCharacterValue(intDigit1).ToString() + GetCharacterValue(intDigit2).ToString();             return s;         }         private static char GetCharacterValue(int i)         {             return (char)(i + 64);         }         private static int GetNumberValue(char c)         {             return (int)c - 64;         }             #endregion         #region Public Properties         private int _intRow, _intCol;         private string _strA1Reference;         public int Row         {             get             {                 return this._intRow;             }         }         public int Column         {             get             {                 return this._intCol;             }         }         public string A1Reference         {             get             {                 return _strA1Reference;             }         }         #endregion         #region Test Wrappers #if DEBUG         public static string TestConvertIntToStringColumn(int intCol)         {             return ConvertIntToStringColumn(intCol);         } #endif         #endregion     } }

Unit Tests:

This kind of class - purely an in-memory data structure with no external dependencies, is easily unit tested. This codes uses the NUnit 2.2 Framework.

using System;using NUnit.Framework;using ExcelUtilities;namespace UnitTests{ /// /// Summary description for TestCell. /// [TestFixture] public class TestCell { public TestCell() { } #region A1 Constructor [Test] public void GetRowAndColumn_1() { Cell c = new Cell("B",3); Assert.AreEqual(3,c.Row); Assert.AreEqual(2,c.Column); } [Test] public void GetRowAndColumn_IgnoreCase() { Cell c = new Cell("b",3); Assert.AreEqual(3,c.Row); Assert.AreEqual(2,c.Column); } [Test] [ExpectedException(typeof(ArgumentException))] public void GetRowAndColumn_InvalidCol() { Cell c = new Cell("ZZ",3); } [Test] [ExpectedException(typeof(ArgumentException))] public void GetRowAndColumn_InvalidRow() { Cell c = new Cell("A",-1); } [Test] [ExpectedException(typeof(System.ArgumentNullException))] public void GetRowAndColumn_InvalidColumn0() { Cell c = new Cell(null,-1); } [Test] [ExpectedException(typeof(System.ArgumentException))] public void GetRowAndColumn_InvalidColumn1() { Cell c = new Cell("",-1); } [Test] [ExpectedException(typeof(System.ArgumentException))] public void GetRowAndColumn_InvalidColumn2() { Cell c = new Cell("#$%",-1); } #endregion #region RC constructor [Test] public void RC_1() { Cell c = new Cell(2,3); Assert.AreEqual(2,c.Row); Assert.AreEqual(3,c.Column); } [Test] [ExpectedException(typeof(ArgumentException))] public void RC_2() { Cell c = new Cell(-1,3); } [Test] [ExpectedException(typeof(ArgumentException))] public void RC_3() { Cell c = new Cell(1,-1); } #endregion #region ConvertIntToStringColumn #if DEBUG [Test] [ExpectedException(typeof(System.ArgumentOutOfRangeException))] public void ConvertIntToStringColumn_1() { Cell.TestConvertIntToStringColumn(-1); } [Test] [ExpectedException(typeof(System.ArgumentOutOfRangeException))] public void ConvertIntToStringColumn_2a() { Cell.TestConvertIntToStringColumn(0); } [Test] [ExpectedException(typeof(System.ArgumentOutOfRangeException))] public void ConvertIntToStringColumn_2() { Cell.TestConvertIntToStringColumn(257); } [Test] public void ConvertIntToStringColumn_3() { Assert.AreEqual("A",Cell.TestConvertIntToStringColumn(1)); } [Test] public void ConvertIntToStringColumn_4() { Assert.AreEqual("IV",Cell.TestConvertIntToStringColumn(256)); } [Test] public void ConvertIntToStringColumn_5() { Assert.AreEqual("Z",Cell.TestConvertIntToStringColumn(26)); }#endif #endregion #region A1Reference [Test] public void A1Reference_1a() { Cell c = new Cell("B",3); Assert.AreEqual("B3",c.A1Reference); } [Test] public void A1Reference_CaseInsensitive() { Cell c = new Cell("b",3); Assert.AreEqual("B3",c.A1Reference); } [Test] public void A1Reference_2() { Cell c = new Cell(3,2); Assert.AreEqual("B3",c.A1Reference); } #endregion }}

There's not much to explain about this code. One note about using the #if DEBUG to enclose a test-wrapper. We could use reflection to avoid this, as explained in this Code Project article, but I was just being quick and informal. Next we'll do the more interesting task of using this in an ExcelReader utility class.

C# 3.0 LINQ - querying data

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/c_30_linq__querying_data.htm]

This looks so cool:

[Quote from MSDN]

The LINQ Project is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. It extends C# and Visual Basic with native language syntax for queries and provides class libraries to take advantage of these capabilities. 

[End Quote]

It looks like you'd have the similar query abilities that you'd have in SQL or XPath, and it beats writing tons of custom code. See 101 examples here: http://www.msdn.microsoft.com/vcsharp/future/linqsamples/

Extending Excel with .Net Part III: Context Menus

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/extending_excel_with_net_part_iii_context_menus.htm]

In my previous blog about extending Excel with .Net, I showed how an Excel sheet could call a .Net console app. This blog will show how to add Context Menus to a specific sheet. This is a little off topic for me as I don't normally discuss VBA. There are certainly other resources to cover this, like http://www.cpearson.com/excel.htm

What we'd like to do is add Context Menus for just a single workbook. When you close the books, the menus disappear, when you re-open it they reappear. Normally when you add an item to the context menu, it persists. So we'll add it when we open the workbook, and remove it when we deactivate the workbook. Something like so:

Private Const m_Context1 As String = "Context1" Private Const m_Context2 As String = "Context2" Private Sub Workbook_WindowActivate(ByVal Wn As Excel.Window)     Call AddItemToContextMenu End Sub Private Sub Workbook_WindowDeactivate(ByVal Wn As Excel.Window)     Call RemoveContextMenuItem End Sub Sub AddItemToContextMenu()     Dim cmdNew As CommandBarButton         Set cmdNew = Application.CommandBars("Cell").Controls.Add     With cmdNew         .Caption = m_Context1         .OnAction = "Method1"         .BeginGroup = True     End With     Set cmdNew = Application.CommandBars("Cell").Controls.Add     With cmdNew         .Caption = m_Context2         .OnAction = "Method2"         .BeginGroup = False     End With End Sub Sub RemoveContextMenuItem()     On Error Resume Next     Application.CommandBars("Cell").Controls(m_Context1).Delete     Application.CommandBars("Cell").Controls(m_Context2).Delete End Sub

A couple notes:

1. We reference the menu items by a string, so I refactor this to a constant such that string doesn't get accidentally mistyped during the remove phase.
2. The WindowActivate and WindowDeactive trigger the events we need. We can't merely just do this on open or close because the user may have two workbooks open at a time, and we don't want our context menu items to appear in the other workbook.
3. The OnAction property is set to the method to be called.

While it's nice to be able to add such menu items, it really makes me appreciate WinForms in .Net!

Next post: How to read the Excel book from .Net.

Excel Migration Part II: Calling a .Net Process

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/excel_migration_part_ii_calling_a_net_process.htm]

In my previous blog I discussed extending Excel with VBA vs. re-writing the app. I said I wanted to create a hybrid solution. Here I'll propose a solution for using .Net to improve how you view and share the final data with clients.

Essentially we want to create a "thin" report of our final data. This report is thin in the sense that it only contains the data and the means to present it. It does not contain calculation logic, validation, user interactions, security, etc... The context is that you've used your Excel sheet to calculate the data needed for a widget, and now you want to send that data to a client. You don't want to send them the entire workbook, as this is:

• Platform Dependent - it requires that they have not just Excel, but the correct version of Excel.
• Bulky - It physically takes up more space (you store all the rows, and all the calculations for each row)
• Messy - while you can format things nicely in Excel (such that it prints okay), this is extra work to do. It is also much harder when you only want to display one row.
• Excess - Suppose you're using the same sheet to manage widgets for 10 different clients, you only want to send one (or several) rows of output - not every row in your sheet.
• Unsecure - Say you only want to send your output, not all the calculations you used. Your calculations may even be intellectual property that can't be distributed. Sure you could put a password on the Excel sheet, but that is an extra step and still gives away your calculations - albeit in a locked format. Wouldn't it be ideal to not even send the calculations at all?
• Changeable - suppose you're sending this to a manager - do you really want to risk that manager "accidentally" fiddling with your calculations or breaking the Excel sheet? Ideally you could give them an idiot-proof document with no "movable parts" that cannot be broken.

One solution that solves all of this is to have a tool generate an Html report from only the Excel output that you want. Html would have the benefits:

• Platform Independent - it has no need for Excel. Anyone in the world can read HTML
• Light weight - a small, text-based file.
• Easily formatted
• Minimal - only send the output you want
• Secure - it only includes the results, none of the proprietary calculations used to get those.
• Unchangeable - a pure Html doc has no "movable parts". It is read only and impossible even for the most clumsy user to mess up.

How we would implement this:

1. Create a .Net Console App that takes in the necessary info to read the Excel book and create the desired Html Report
2. In Excel, have a processes to call this app via the command line.

Say this .Net console app, called CreateHtmlReportConsole,  would take four parameters: the physical path to the Excel book, the physical path to an html file that servers as a template for the report, the Excel worksheet, and the Excel row to pull data from. A sample command line may look like so (I've included the quotes in case your paths have spaces in them):

"C:\myProjects\CreateHtmlReportConsole.exe" "C:\ExcelBooks\Book1.xls" "C:\myProjects\Template.htm" 1 14

I'll show step #2 here, and discuss step#1 more in another blog. You could have a method in VBA (from a button or context menu) use the Shell command to call an external process like our .Net console app. For example:

Dim strCmd As String Dim strCurrentDirectory As String Dim strThisFileName As String 'Get current directory and filename: strCurrentDirectory = ThisWorkbook.Path & "\" strThisFileName = ThisWorkbook.Name 'Create a command line like: ' "C:\myProjects\CreateHtmlReportConsole.exe" "C:\ExcelBooks\Book1.xls" "C:\myProjects\Template.htm" 1 14 strCmd = strCurrentDirectory + "CreateHtmlReportConsole.exe """ + _     strCurrentDirectory + strThisFileName + """ """ + _     strCurrentDirectory + "Template.htm"" 1 " & intRow Shell (strCmd)

Next step: how add dynamic context menus in Excel, from which you could call this code.

Migrating an app from Excel to .Net, Part I

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/migrating_an_app_from_excel_to_net_part_i.htm]

I recently had the challenge of improving upon an Excel workbook. This was a complicated sheet, with 200 columns of ugly calculations that solved a difficult engineering problem. There were many rows, one for each widget being engineered. I'm going to blog about several issues I encountered for this.

The first issue I needed to deal with: should I extend Excel (with VBA) or rewrite the application in .Net. I see the following pros and cons:

	Extend Excel with VBA	Rewrite in .Net
Pro	Potentially much quicker - not rewriting all the complex calculations Excel is already designed for this kind of thing. It's easy to copy new rows, and it has a built in tools like GoalSeek (a numeric solver) of which there is no clear counterpart available in .Net (perhaps I could implement Newton's Method with numeric derivatives, but that takes some work would be very error prone for me to mess up). Excel is easy to maintain for the end user, especially if they need to make a change.	Powerful platfom Easier to test code with Test-Driven-Development (what if you need to modify a formula in your Excel book, and you want to make sure everything still works) Splits out input, logic, and view into separate components. For example you could have a WinForm take (and validate) the input, a backend DLL could do all the logic, and then you could generate an Xml file or Html document with the final output results.
Con	Limited power, VBA is messy to deal with compared to .Net with C#, Visual Studio, and the .Net Framework Excel combines the input, logic, and view all into one sheet. For example is you wanted to show the client only the calculated output, you'd either send them the entire sheet (which lets them see your calculations unless you do some fancy cell-hiding), or copy those values to a new document.	Lot of extra work Hard for the end user to change or maintain

The final solution I'm looking at: make a hybrid. Continue using Excel for user input and calculations, extending it with VBA. But use .Net to read the Excel document and generate output views. I'll discuss this more in my next blog.

10 Tips on CodeSmith SubTemplates, Batch Mode, and XmlProperties

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/10_tips_on_subtemplates_batch_mode_and_xmlproperties.htm]

I've finished using CodeSmith to help with generating some of our tiers (or should I say "just started"?). So I posted some things I've learned from it: 10 Tips on SubTemplates, Batch Mode, and XmlProperties on the CodeSmith Peer Support Forum. This covers solutions I've found for:

1. Passing values between subtemplates
2. Handling that Sub template does not inherit parent's merge strategy
3. Handling that target of RenderToFile must exist
4. Handling Multi-level subtemplates
5. Throwing and using errors in batch mode
6. Embed XmlProperty filename instead of data
7. Dynamically make your own batch file
8. Only write new content
9. Reusable code for XmlProperties
10. Instantiate an XmlProperty at runtime

Details on the forum.

Using Sysinternal PsTools to find what has locked that process

[This was originally posted at http://timstall.dotnetdevelopersjournal.com/using_sysinternal_pstools_to_find_what_has_locked_that_proce.htm]

Every get one of those frustrating errors "Cannot access file because it is being used by another process"? For example, you may be trying to delete or rename a file, but this error message won't let you. Sometimes it is as simple as closing an application that obviously has locked that file, but sometimes it's just more complicated - perhaps some service (which has no GUI, and therefore may be less intuitive) has locked the file. Ideally you could just click the file in Windows Explorer and it would tell you what locked the file and what to do.

There is a solution - Sysinternals provides a free tool, Process Explorer, that lets you find all the processes accessing a file. The tools requires no MSI; you can directly copy it to your machine and run it. It's like Task Manager on steroids (in a good sense).

Initially one may make sure that they don't have the file open in some sort of editor, but the problem may persist even if everything is closed. Go to Find > Find Handle, and type in the name or directory of the file. Click search, and it will show you all processes accessing that file. You can now kill those processes such that you can delete the file.

This is actually very educational - you'll see that IIS, other instances of Windows Explorer, certain background services, or apps that you wouldn't think of, all can lock files.