Pattern of the week: Memento

January 14, 2012 by · Leave a Comment
Filed under: Architecture, Design, Patterns, Programming 

This weeks pattern is the Memento Pattern. We will spend this week reviewing this pattern, how it is used and what its intent is for. For some more in-depth definition you can see  here and here are some useful descriptions, but to appreciate this we will review the intent and possible solutions and applications for this pattern in everyday work.

Here is the intent of the Memento pattern: “Without violating encapsulation, capture and externalize an object’s internal state so that the object can be returned to this state later.  A Magic cookie encapsulating check point capability, promote undo or rollback to full object status.”

 

memento

Example application for this pattern has been to support undo or rollback, other common implementations are a finite state machine.  In my own personal development projects I have used memento pattern to create and audit trail with full play-back of the payloads as an encapsulation exercise. 

NCover 1.5.8 Errors (profiler connection not established or [ncover] Index was outside the bounds of the array)

January 12, 2012 by · Leave a Comment
Filed under: Build, Programming 

I am creating a NANT and NCover build that is braking randomly, the tests and bulid succeed, but the nany scripts break with profile error, then once I fixed that it was an array out of bounds error.)

The profile connection error was related to elevated permissions on Vista or windows 7, once you run build script as admin or remove UAC this resolved the issue.  The correct approach on this would be to dig deeper into the identity and permissions the build script has to execute and ensure this is covered.

 

The issue with index out of bounds, appears to be a bug with NCover 1.5.6 or greater.  If you grab NCover 1.5.5 it appears to work without issue and this resolved my problems. You can download NCover 1.5.5 here:

NCover 1.5.5 Download!

Happy Scripting.

Skyline Query in T-SQL

January 5, 2012 by · Leave a Comment
Filed under: Programming, SQL 

Here is the sample database schema as an example to illustrate the procedure and process of implementing a Skyline Query that works with Microsoft Sql Server, this should work with all SQL Server that support nested queries. 

 
 
CREATE TABLE [dbo].[Housing](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [DistanceFromWork] [numeric](13, 4) NOT NULL,
    [Price] [money] NOT NULL,
    [SquareFootage] [int] NOT NULL,
    [Name] [nvarchar](50) NOT NULL,
    [City] [nchar](10) NULL,
 CONSTRAINT [PK_Housing] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, 
ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]
 
 
SET IDENTITY_INSERT [dbo].[Housing] ON
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (1, CAST(1.2000 AS Numeric(13, 4)), 320394.0000, 1600, N'Brooke', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (2, CAST(8.8000 AS Numeric(13, 4)), 179000.0000, 1924, N'Mica', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (3, CAST(1.1000 AS Numeric(13, 4)), 280000.0000, 1800, N'Blackbone', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (4, CAST(12.2000 AS Numeric(13, 4)), 180000.0000, 1923, N'Vienna', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (5, CAST(0.1000 AS Numeric(13, 4)), 500000.0000, 1200, N'Zappod', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (6, CAST(14.2000 AS Numeric(13, 4)), 88000.0000, 3200, N'Rancher', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (7, CAST(1.3000 AS Numeric(13, 4)), 310000.0000, 2300, N'Joomba', N'Boston    ')
INSERT [dbo].[Housing] ([ID], [DistanceFromWork], [Price], [SquareFootage], [Name], [City]) 
VALUES (8, CAST(0.9000 AS Numeric(13, 4)), 312222.0000, 1900, N'Dumbo', N'Boston    ')
SET IDENTITY_INSERT [dbo].[Housing] OFF
 

Here is the query that will do a three-dimensional skyline query here is an output before and after.

------listing for compare------
SELECT *
  FROM [Test].[dbo].[Housing]
order by distancefromwork, price
 
--------optimization query---------
 
    SELECT *
FROM [Housing] h
WHERE h.city = N'Boston' AND NOT EXISTS(
SELECT *
FROM [Housing] h1
WHERE h1.city = N'Boston' AND h1.DistanceFromWork <= h.DistanceFromWork AND
h1.price <= h.price AND  h1.SquareFootage >= h.SquareFootage and
(h1.DistanceFromWork < h.DistanceFromWork OR h1.price < h.price OR h1.squarefootage > h.squarefootage))
order by distancefromwork, price 

 

The performance of this is not very good as it cannot be easily optimized since you are nesting queries to compare and remove other records that are dominated by the ones that you care about. 

BEFORE

ID    DistanceFromWork    Price    SquareFootage    Name    City
5    0.1000    500000.00    1200    Zappod    Boston    
8    0.9000    312222.00    1900    Dumbo    Boston    
3    1.1000    280000.00    1800    Blackbone    Boston    
1    1.2000    320394.00    1600    Brooke    Boston    
7    1.3000    310000.00    2300    Joomba    Boston    
2    8.8000    179000.00    1924    Mica    Boston    
4    12.2000    180000.00    1923    Vienna    Boston    
6    14.2000    88000.00    3200    Rancher    Boston    

 

AFTER

ID    DistanceFromWork    Price    SquareFootage    Name    City
5    0.1000    500000.00    1200    Zappod    Boston    
8    0.9000    312222.00    1900    Dumbo    Boston    
3    1.1000    280000.00    1800    Blackbone    Boston    
7    1.3000    310000.00    2300    Joomba    Boston    
2    8.8000    179000.00    1924    Mica    Boston    
6    14.2000    88000.00    3200    Rancher    Boston    

As you can see it removed item # 1 and #4, as these were dominated by other properties in the system.

 

I will explore this in the future to look at other methods for achieving similar results, getting what you really want out of a system.

Skyline Query

January 4, 2012 by · 1 Comment
Filed under: Design, Programming 

A Skyline query is one of the most useful but least obvious data programming patterns.  When you have a set of data and a query with 2 constraints that are conflicting, the output of this is the “Skyline” Query.  If you want to thing about what skyline means and why it describes this issue take the example of searching for cars to drive, and lets say you want the biggest car with the best gas mileage.  These two constraints are conflicting since as cars get bigger, gas mileage usually goes down, but a skyline query essentially runs this query across a set of data and then grabs the tops of the skylines. This would take a cluster of data and grab the top 2 or 3 in that set, then move along to find the next set, this way you can give the user choices without having to come up with an artificial weight or

 

A more realistic example would be searching for a house that is the cheapest cost, that is the biggest, and say closest to where you work.  This is one of the rare data patterns that has so many real world applications it is not commonly known below we will show an implementation of an efficient algorithm shown in this paper to offer stellar performance.

The Skyline Operator

They showed that the best performance for a multi-dimensional skyline processor is a Block – Nested Loop.

Here is some psudo-code with implementation coming next. I’ll show brute-force then also

an implementation with the block-nested loop for T-SQL..

   1:  M – Input of dimensional points
   2:  R output of dimensional points
   3:  T temporary file for dimensional points
   4:  S set of dimensional points
   5:  p < q point p is dominated by point q
   6:   
   7:  function SkylineBNL(Input)
   8:  {
   9:  Output = 0; 
  10:  Temp = 0; 
  11:  Set=0;
  12:  CountIn = 0;
  13:  CountOut = 0;
  14:   
  15:  while(input)
  16:  do
  17:  {
  18:  foreach p in Set
  19:  {
  20:  if(TimeStamp(p)=countin 
  21:  {
  22:  save(Output, p), Release(p)
  23:  }
  24:  Load(Input,p)
  25:  TimeStamp(p) = CountOut;
  26:  CountIn++
  27:  foreach(q in Set)
  28:  {
  29:  if(p>q) release (p) break;
  30:  if(p<q) then reelase(q);
  31:  }
  32:  if(MemoryAvailable)
  33:  {
  34:  save(Temp,p), release(p)
  35:  CountOut ++;
  36:  }
  37:  //keep it running to iterate through entire sets.
  38:  if(Main == empty)
  39:  {
  40:  Main = Temp;
  41:  Temp=0;CountIn=0;CountOut=0;
  42:  }
  43:  }
  44:  }
  45:   
  46:  //flush
  47:  foreach(p in Set)
  48:  {
  49:  save(Output,p), release(p)
  50:  }
  51:   
  52:  return Output;

WCF 3.5 SP1 allows for circular references without adding any code

April 7, 2009 by · 1 Comment
Filed under: Programming, Software Configuration, WCF 

WCF now provides a way to avoid circular reference problems on serialization, which before required you to initialize the Data Contract Serializer with a KeepReferences setting. Another option was to cheat and use a NetDataContractSerializer which wasn’t easier without cusotm behaviors or special code, now it is as simple as adding an attribute.

 Here is one such exception you may receive if you are having issues:

“Object Graph for Type X Contains Cycles and

Cannot be Serialized if Reference Tracking is Disabled”

 

Example that will work wonders without anything else required.

[DataContract(IsReference = true)]

    public class Person

    {

        [DataMember]

        public string Name

        { get; set; }

       

        [DataMember]

        public string Address

        { get; set; }

 

  [DataMember]

        public List<Person> Children

        { get; set; }

 

    }

Now it works, Another example more appropriate is using eXtensible Access Control Markup Language (XACML)  specification.

[DataContract(IsReference = true)]

    public class PolicySet

    {

        [DataMember]

        public string Name

        { get; set; }

       

 

  [DataMember]

        public List<PolicySet> PolicySets

        { get; set; }

 

  [DataMember]

        public List<PolicySet> ParentPolicySet

        { get; set; }

 

 

    }

[DataContract(IsReference = true)]

    public class Policy

    {

[DataMember]

        public string Name

        { get; set; }

 

        [DataMember]

        public PolicySet ParentPolicySet

        { get; set; }

 

  [DataMember]

        public List<Rule> Rules

        { get; set; }

 

 

    }

[DataContract(IsReference = true)]

    public class Rule

    {

[DataMember]

        public string Name

        { get; set; }

 

        [DataMember]

        public Policy ParentPolicy

        { get; set; }

 

  [DataMember]

        public List<Targets> Targets

       { get; set; }

    }

Serializing types that return interfaces instead of instances in WCF 3.5 SP1

April 7, 2009 by · Leave a Comment
Filed under: Programming, Software Configuration, WCF 

I came across interesting problems with WCF serialization specifically the DataContractSerializer and limitations of the runtime.

Say you have a class like so

[Serializable]

    public class Service

    {

        public IList<Person> People

        {

            get; set;

        }

    }

   

    [Serializable]

    public class Person

    {

        public string Name

        { get; set; }

       

        public string Address

        { get; set; }

    }

 

 

 

 If you try to get this working in a WCF service it won’t work, basically the problem is that you are returning an interface, not a concrete type, and the serializer doesn’t know how to deal with it. To allow this to work you basically have to implement a data contract for the types invovled, then it seems to understand, and also adding a KnownType attribute helps it to serializer things without getting generic random errors on the service.

    [ServiceContract]

    [KnownType(typeof(Person))]

    public class Service

    {

        [DataMember]

        public IList<Person> People

        {

            get; set;

        }

    }

 

    [DataContract]

    public class Person

    {

        [DataMember]

        public string Name

        { get; set; }

       

        [DataMember]

        public string Address

        { get; set; }

    }

 

 

Another common issue is that best practices say your shouldn’t have setters on a collection, but let users add to it or remove, from the collection itself.   the WCF serializer requires read/write access to the collection for its duty, so a way around it is to do this

    [ServiceContract]

    [KnownType(typeof(Person))]

    public class Service

    {

        [DataMember(Name = “People”)]

        private List<Person> people = new List<Person>();

 

       

        public IList<Person> People

        {

            get

            {

                return people;

            }

        }

    }

   

    [DataContract]

    public class Person

    {

        [DataMember]

        public string Name

        { get; set; }

       

        [DataMember]

        public string Address

        { get; set; }

    }

WCF 3.5 SP1 issue with service to service authentication error (caller not authenticated)

April 6, 2009 by · 3 Comments
Filed under: Programming, Software Configuration, WCF 

I recently discovered an issue with WCF 3.5 SP1, specifically the error is “The caller was not authenticated by the service”

System.ServiceModel.Security.SecurityNegotiationException was unhandled
  Message=”The caller was not authenticated by the service.”

My setup was using WSHttpBinding with message security, and it worked locally but not on a hosted machine in a domain.  I had two services on same box, calling each other, and this issue cropped up.  I was using the dns as the identity in the configuration file, however there was a change in 3.5 SP1 and now it requires either a SPN (service principal name) or a UPN (user principal name) to work correctly.  The interesting thing is you don’t even need to know the spn since it is automatically created when the service is hosted.  This is only needed for client configuration calling the services.

You can leave SPN blank it will work if default is kerberos, otherwise you will want to specify the specific spn for negotiate to fall back on.

So, if the host is running with user credentials, you should use its UPN:

<identity>
<
userPrincipalName value=user@example.com” />
</identity>

And if the host is running as s service, specify the SPN

<identity>
<
servicePrincipalName value=Host/MACHINENAME” />
</identity>

I hope this helps.