IdeaBlade DevForce – Model Setup Walk-through – Background

I know the world is about to change in April when VS2010 goes live, however, we are still using VS2008 and will be for at least a few more weeks.  In the hopes that this walk-through provides some insight to someone, even if it is just me, here goes.

Background

We are  creating a Silverlight application for several reasons:

  1. Web development is not nearly complex enough, so we sought out a unique challenge involving the newest, least documented, voted most-likely-to-be-completely-re-done-in-the-next-version technology (excepting WWF, I mean, WF, which will probably hold that distinction for years to come).
  2. Our users do everything in Excel, meaning that much of the data entry UI for this app needs to be as Excel-ish as possible, including the dreaded Multi-Add (adding multiple rows between saves) and Multi-Edit (editing multiple rows between saves).  [FYI: It turns out that none(?) of the OTC Grids are actually designed for this.  You can make it work with Telerik if you want to try hard.  See postings like this one in Telerik's forums for more info.  They have come a long way since that posting.]
  3. Management is opposed to fat client or virtualization, so the app has to be web-based.
  4. After much trial-and-error, and trial-and-failure, and trial-and-compromise, we deployed an ASP/Ajax solution that was less than satisfactory and not-at-all Excel-ish, so we had to find a “better way”.
  5. We have a code base that comprises 8 years of work in VB 6, classic ASP, ASP.Net 1, ASP.Net 2, ASP.Net 3.5, VB.Net (all sorts), Infragistics (an old version), Telerik (several versions), copious quantities of javascript, several generations of CSS and DHTML, including many of the really exotic first generation tricks, Ajaxified ASP pages, ASP/Ajax pages and an Oracle back-end, so we knew that ASP was not going to give us what we needed.
  6. None of us had ANY desire to explore Flex.
  7. We are a Microsoft shop, so building a Java app was not really an option.
  8. Silverlight 3 was coming and promised enough features for us to begin writing an Enterprise app.  (It turns out that we jumped the gun on this, but we were not alone and SL4 promises to remedy many of the “it’s-not-ready-for-the-enterprise” issues.)

We are using DevForce and the Entity Framework because:

  1. Having worked with Hibernate, NHibernate, and SubSonic (which I preferred over NHibernate), I was convinced that a Data Layer / ORM would make our application specific code much easier to write and would provide more and better infrastructure/plumbing than we ever could.
  2. Having been told:  “No Open Source”  and given 3 weeks to pull SubSonic out of a working project and replace it with a roll-my-own ORM, I knew that many features we needed (Concurrency and caching, to name just two) were going to be a HUGE effort to write myself, and that if this was a Make or Buy decision, that Buy was clearly the better choice.  (Check out Davy Brion’s Build Your Own Data Access Layer Series for a deeper examination of the Make option.)
  3. Having been exposed to DevForce Classic a few years ago, I knew their product provided much of what we needed, and they were on the cusp of releasing a Silverlight version of their WPF framework.  As an added bonus, their documentation, and Ward Bell’s blog, are highly readable and provide excellent project guidance and design philosophy.
  4. DevForce sits on top of Entity Framework, which does not support Oracle, but as this was about to kill the deal, we found that DevArt’s Oracle drivers were finally supporting EF properly.

We are using Prism and Unity because:

  1. If you’re gonna do this thing, you might as well go all in.
  2. Having worked with CAB for a Windows App, I understood the potential of a component based application framework — or at least imagined I understood it, can anyone really understand anything PnP publishes?
  3. After dabbling in the Java world for a few months, I had developed an appreciation for Spring, Dependency Injection, configuration over coding and convention over configuration.
  4. I am unrepentant about preferring Agile development practices, and many of the “best practices” prescribed by PnP and implemented in Prism, represent many Agile coding principles in action.  Both VersionOne and RallyDev have Agile 101 documentation on-line.   Mike Cohn has written several excellent books introducing teams to Agile practices.

So we ended up with the following technology stack:

  1. Prism 2 for its Modular framework
  2. Unity for Dependency Injection
  3. Microsoft Silverlight 3 for the UI
  4. Telerik for Silverlight for several of its UI components (alas, we still have to do far too much ourselves in this area though)
  5. IdeaBlade DevForce for Silverlight for its Silverlight friendly Business Entity model and asynchronous client-server communication layer
  6. DevArt dotConnect for Oracle drivers for its support of the MS Entity Framework and it’s excellent Entity Developer tool
  7. Microsoft Entity Framework for the server-side Entity Model and database connectivity

Mysteries of Silverlight Deployment – Error Code 2104 – Could not download the Silverlight application

So I’m trying to open a newly deployed Silverlight site/app in my browser.  I have the latest version of Silverlight, the files are deployed, the security is configured.  Still, one of my Silverlight apps is downloading to the browser properly, the other one, gives me the following error:

Error: Unhandled Error in Silverlight Application
Code:  2104
Category: InitializeError
Message:  Could not download the Silverlight application.  Check web server settings

I checked the web.config.  Nothing different between the site that works and the one that doesn’t.

I checked a bunch of other things.  Nada.

I finally broke down and started a tab-by-tab comparison of the Web Site Properties dialog in IIS Manager.  (IIS 6.0)  On the HTTP Headers tab, everything matched.  Then I clicked on the MIME Types… button, and there was my discrepancy.

In order for IIS 6 to serve up a Silverlight App, you have to add the following MIME Types to the Web App Properties, HTTP Headers tab:

.xaml application/xaml+xml
.xap application/x-silverlight-app
.xbap application/x-ms-xbap

After adding these entries, I had to click OK a couple of times to get back to IIS Manager and save the change.  Then I had to close and re-open my browser because IE likes to cache everything so even a refresh didn’t download the Silverlight app.  Once I re-opened, the Silverlight app loaded, and I was back to debugging.  Yay.

Moving Code from the Developer’s Integration Environment to the QA Environment

When we’ve completed coding (and, if we’re feeling especially adventurous, testing said code), we promote our locally developed code to the Dev server. This serves as our integration environment. CruiseControl.Net watches our SCM for any changes promoted to the DEV branch, then it:

  1. Updates the configuration files to point to the Dev server instead of the developer’s local workstation;
  2. Builds our solution in Debug mode (for more verbose logging during the unit test runs);
  3. Runs the unit tests;
  4. Builds the solution again in Release mode;
  5. Runs the unit tests again to ensure they work in Release mode too; and
  6. Finally, publishes the necessary files to IIS.

Today, I need to figure out how much has to change to get this project onto our QA server.

The Problem

Pointing the QA environment to the QA database.

The biggest difference between the process described above and the process to move to QA is going to be changing the back end database that the solution points to. I.e. The Developer’s local workstation and the Dev server both point to the Dev database. (It’s an Oracle back end, so we do not have a separate local and server database — even if we wanted to, but I’m not sure we would.)

ORACLISM

An aside about Oracle schema ownership. As a SQL Server developer, there are several Oraclisms that confound me. I’m not sure that this is one of them, but here’s my take on it: In SQL Server, most databases are created by dbo and most tables are created as dbo. Security is then applied to users and groups to grant access to various tables. If you need to create a Dev database and a QA database for the Production database named AccountingSystemDB, you might create a new database, again as dbo, called AccountingSystemDEV and another called AccountingSystemQA. In Oracle, this notion of separate databases is superseded by a notion of separate schemas WITHIN a single database. There is no obvious corollary to dbo. The Oracle system user serves a very different purpose, perhaps relating more to the notion of sa in SQL Server. Instead, an Oracle DBA will create a Schema Owner (i.e. a user?) and then create the Schema as that owner, granting rights to that Schema to users.

For example, the Oracle Server has a single Database instance, within which, you will see three schemas named ACC_SYS_DV, ACC_SYS_QA and ACC_SYS_P. Within each schema, you will find that schema’s version of the tables. I.e. ACC_SYS_DV.ACCOUNT, ACC_SYS_QA.ACCOUNT and ACC_SYS_P.ACCOUNT. Also, a lot of database apps, use a single Application User Id to access the database, relying on the Application to implement security. (Or at least, ours does.) For each of the three environments, this Application User logs in to one of the three Schemas.

I am concerned that Entity Framework knows the Database and Schema in the connectionstring in the app.config for the ModelEF project:

 ...
 <edmKeys>
  <edmKey
   name="DataSource"
   connection="metadata=res://ModelEF/ModelEF.csdl| res://ModelEF/ModelEF.ssdl| res://ModelEF/ModelEF.msl; provider=Devart.Data.Oracle; provider connection string=" User Id=APPDV; Password=*****; Server=devTns.world; Home=oraodac11g_home1;Persist Security Info=True""
   containerName="ModelEF.Entities">
   ...
  </edmKey>
  ...
 </edmKeys>
</ideablade.configuration>

Above we see the Oracle server instance devTns.world, and the Dev database schema owner APPDV. There is also a Home= entry pointing to the developer’s OraHome that needs to be removed.

EF then maps the Schema owner into the EDML (EDMX) file:

...<EntitySet
  Name="ACCOUNTs"
  EntityType="ModelEF.Store.ACCOUNT"
  store:Type="Tables"
  Schema="APPDV" <=NOTICE DV SCHEMA OWNER IS HERE
  Table="ACCOUNT" />...

If this was a SQL Server solution, the schema owner would be the same (whether it is dbo or not) in the AccountingSystemDEV and AccountingSystemQA databases. In Oracle, this is not the case, so I am curious if I will need to update the Schema tag in the EDML (EDMX) . If so, the promotion to QA will force a rebuild, not just a reconfigure.

The Ideablade configuration also knows the Database and Schema in the connection attribute of the edmKey tag in the Model project’s app.config and in the web host project’s web.config:

<configuration>
 <connectionStrings>
  <add name="Model.EFConnectionString"
   connectionString="metadata=res://*/ModelEF.csdl| res://*/ModelEF.ssdl| res://*/ModelEF.msl; provider=Devart.Data.Oracle; provider connection string=&quot; User Id=APPDV; Password=*****; Server=devTns.world; Home=oraodac11g_home1; Persist Security Info=True&quot;"
   providerName="System.Data.EntityClient" />
 </connectionStrings>
</configuration>

[It turns out, I do need to change the tags, change the connection strings, and regenerate the pregenerated views.]

A SOLUTION

Since we are using CruiseControl.Net (CCNet) for our automated build, and since we are using Nant for some of the less “build-ish” and more “configure-ish” tasks, I was able to add a Nant task in the CCNet project configuration that does a string replace in all the EDML (EDMX) and .config files, and then uses the T4 Template technique to recreate the Pregenerated Views.

In order to use the T4 Template attached to the ADO .Net blog, I had to make two changes because we are using Devart Entity Developer. First, the .tt file needs to be renamed from ModelEF.Views.tt to ModelEF.PregeneratedViews.tt. Second, the following line in the .tt file has to be changed:

string edmxFileName = Path.GetFileNameWithoutExtension(this.Host.TemplateFile).ToLowerInvariant().Replace(".views", "") + ".edmx";

Should be replaced with:

string edmxFileName = Path.GetFileNameWithoutExtension(this.Host.TemplateFile).ToLowerInvariant().Replace(".pregeneratedviews", "") + ".edml";

Here is the CCnet task in the CruiseControl.Net ccnet.config file to execute the Nant task:

<project name="PromoteToQA">
 ...
 <tasks>
  <nant>
   <buildFile>ChangesBeforeBuild-QA.build</buildFile>
   <executable>E:\Program Files\nant-0.86-beta1\bin\nant.exe</executable>
  </nant>
 </tasks>
 ...
</project>

And here is the Nant task itself:

<project name="ChangesBeforeBuild-QA" default="run">

 <target name="run">
  <call target="change Database" />
 </target> 

 <target name="change Database">
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.edml" />
     <include name="**/app.config" />
     <include name="**/web.config" />
     <include name="**/*EF.Designer.cs" />
     <include name="**/*EF.edps" />
    </items>
   </in>
   <do>
    <attrib file="${filename}" normal="true" />
    <loadfile file="${filename}"
     property="file.textContent">
     <filterchain>
      <replacestring from="APPDV" to="APPQA" ignorecase="true"/>
      <replacestring from="*****" to="*****" />
      <replacestring from="devTns.world" to="qaTns.world" ignorecase="true"/>
      <replacestring from="Home=oraodac11g_home1;" to="" />
     </filterchain>
    </loadfile>
    <echo message="Writing Database Change to: ${filename}" />
    <echo file="${filename}">${file.textContent}</echo>
   </do>
  </foreach>
 </target>
 
 <target name="Recreate Precompiled Views Files">
  <!-- unlock the view files -->
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.PregeneratedViews.cs" />
    </items>
   </in>
   <do>
    <attrib file="${filename}" normal="true" />
   </do>
  </foreach>
  <!-- transform the *.PregeneratedViews.tt files to recreate the *.PregeneratedViews.cs files -->
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.PregeneratedViews.tt" />
    </items>
   </in>
   <do>
    <echo message="Transforming: ${filename}" />
    <exec program="dependencies\TextTransform\TextTransform.exe">
     <arg value="${filename}"/>
    </exec>
   </do>
  </foreach>
 </target>
 
</project>

And that’s it. Before CruiseControl.net starts running the actual tasks, it runs this Nant task to prepare the source files for the QA environment.

Why does he keep saying EDML instead of EDMX?

In the Microsoft Entity Framework (EF), the GUI for the Entity Model is called the Entity Designer.  When you create an Entity Model, the model itself is stored in an XML file named model.EDMX.

The EDMX file is: 

a) an XML representation of the Entities and Relationships in your Database (the Storage Model, or SSDL), your application (the Conceptual Model, or CSDL), and the mapping from one to the other (the  Mapping Specification, or MSL); and

b) a repository for the metadata that defines the Entity Designer Diagram layout.

And this works out just fine for SQL Server.  Microsoft provides a nice little driver that is EF compatible, they provide a nice little GUI to create your Model with, and SQL Server tables — if named “normally”, i.e. Customer, Account, etc. — map beautifully to your object model.

This is NOT the case for Oracle.  Oracle still holds its nose whenever anyone says “Microsoft”, so they haven’t, and won’t, provide a driver for EF.  Heaven forbid they should make it easy to use Visual Studio and the .Net framework to work with their data.  Nope — Java, Eclipse, and the Oracle Application Framework are their preferred toolset.

So what is a .Net developer to do?  How can one create an EF model from an Oracle database?

ANSWER:  DevArt

DevArt has gone to the trouble of creating ADO.Net drivers for a variety of databases, including SQL Server and, da-da-da-da, Oracle.

I have been using DevArt’s dotConnect for Oracle drivers for EF purposes for about a year now.  In the early days, they were quite quirky.  I posted to their forums regularly.  Every single problem, suggestion, requirement or wish list item that I posted has been implemented, usually in their next release.  (They post quiet releases every 4 to 6 weeks.  Yay for Agile Development.)  Their latest version (5.35.79 as of this writing) works flawlessly for my purposes.

Anyway, included with their dotConnect drivers, is the DevArt Entity Developer.  This tool ROCKS!  Over the past year, it has matured to the point that I have completely abandoned using the MS Entity Designer for anything.  Instead, I create  and maintain my Entity Model’s using the DevArt Entity Developer. 

Anyway, in order to not step on anyone’s toes, DevArt saves the Entity Model as an EDML file, not an EDMX file.  (Yes, this is the “exact same” Entity Model XML that MS saves.)  They also store a bunch of metadata and project selections in a set of generated files behind the EDML file.  And, as of December, you can select to have the Entity Developer automatically generate the precompiled views file too.  (MS doesn’t do that and provides a 30-step process to accomplish it — A problem that has spawned many web posts.)

One of my favorite features of the DevArt Entity Developer over the MS one, has to do with an Oraclism that just bugs me. 

ORACLISM
Oracle DBA’s, (by tradition, by culture, or by requirement, I’m not sure which), often name database objects in all caps with underscore delimiters.  For example, a table named CustomerAccountType in SQL Server, will probably be named CUST_ACCT_TYP in Oracle.  (There’s an unreasonably low limit to how long the name can be in Oracle, so the DBAs often have to truncate or compress names to make them fit.) 

So, if you have a table defined thus:

CREATE TABLE  CUST_ACCT_TYP (
CUST_ACCT_TYP_ID NUMBER(10) NOT NULL,
TYPE_DESCRIPTION VARCHAR(20) NOT NULL)

Now, you drag and drop that table into the MS Entity Designer*, you will get an object defined thus:

Class:     CUST_ACCT_TYP
Property:  CUST_ACCT_TYP_ID, Double
Property:  TYPE_DESCRIPTION, String

Not a very “.Net-ish” way of defining a class, is it?  Also notice the NUMBER(10), an integral number type, is mapped to a Double.

DevArt’s Entity Developer, however, does a grand job of converting all those names, so you end up with an object defined thus:

Class:     CustAcctTyp
Property:  CustAcctTypId, Int64
Property:  TypeDescription, String

Notice how it even got the data type right for the Id. 

Now, if only I could UPDATE my model (instead of recreating the affected parts of it) when I change the Tables and Views in Oracle… :)

* By the way, you still need the DevArt drivers to do  this.

When Microsoft ate Winternals, where did Disk Commander go?

So I’m trying to make a recovery disk using Bart PE.  My Windows Vista (I know, I know, mock me later) boot partition is HOSED and I can’t get in to Windows.  So I thinks to meself, “this ’ere Bart PE ’as a plugin fer Disk Commander, I wonder if’n that’ll help.”  Turns out Disk Commander used to be sold by Winternals.  (You probably knew that already.)  If you google for it, you will find a zillion links to all sorts of versions that all take you here.  Yay.  Not helpful.

So, reading “Customers looking for desktop diagnostic and recovery tools,” I dutifully click the link, which takes me to the Microsoft Windows Enterprise home page.  Not helpful.

If I had only read the second half of that sentence, I would have saved myself considerable grief:  “Customers looking for desktop diagnostic and recovery tools similar to Administrator’s Pak should refer to the Microsoft Desktop Optimization Pack.”

In fact:  Disk Commander is part of the Microsoft Desktop Optimization Pack (MDOP).  Actually, it’s part of the Microsoft Diagnostics and Recovery Toolset (DaRT) which is part of the MDOP, and it is called ERD Commander.

There’s a very Windows 7-ish TechNet site devoted to MDOP.  Which didn’t help me much because I needed the Vista version.

It turns out that the XP, Vista and Win7 versions of MDOP are all available for “trial” in MSDN.  Listed as “Desktop Optimization Pack” in the “Servers” category.  (As of this writing.)

I think the MDOP 2007 is XP, MDOP 2008 is Vista and MDOP 2009 is Win7.

Woohoo.  Mischief managed.

Follow

Get every new post delivered to your Inbox.