I came to the programming world by way of an Apple II, TI 99/4A, BASIC, VB/VBA and on to C#. But my academic experience is in the world of economics, not computer science, so occasionally I come across things that are really basic (I assume) which I have clearly forgotten somewhere between Apple Basic, HS Electronics, and the ensuing decades of other trivia crammed into my brain.

The Bitwise AND is the operator that left me feeling stupid today.

Many times in my career I’ve come across code to test whether a number is odd or even. Usually I have done or seen modulo division by 2 and a comparison of the remainder.

public static bool IsOdd(int i)
{
    return ((i % 2) == 1);
}

Today I came across

public static bool IsOdd(int i)
{
    return ((i & 1) == 1);
}

I had no idea why that worked. I looked at the ms help

“Binary & operators are predefined for the integral types and bool. For integral types, & computes the bitwise AND of its operands. For bool operands, & computes the logical AND of its operands; that is, the result is true if and only if both its operands are true.”

Not particularly enlightening, so I dug a little deeper. Reviewing binary counting I made a table on a 3x5 card,assuming a 4-bit number for simplicity: 

Decimal 0 1 2 3 4 5 6 7 8
Binary 0000 0001 0010 0011 0100 0101 0110 0111 1000

Then I started thinking about the operation, but I initially processed it in my head as a binary addition not as the proper bitwise operation:

  (3 + 1) 0011 +0001 ----------- =0100 

But that was clearly not right, since the result is 4, and for any positive integer greater than 0 the test would fail if addition was the underlying action. I looked up a couple more definitions for bitwise AND and finally understood this:

A bitwise AND operation (a & b) returns a one in each bit position for which the corresponding bits of both a and b are positive.

The result is a logical test for each bit:

(3 & 1)
0011
&0&0&0&1
=0001

Since the binary representation of decimal 1 is 0001, (front padded with as manyzeros as necessary to match the bit size of the number), the only way the operation ([any integer] & 1) will return 1 is if the rightmost bit is one -- all odd numbers.

The following table describes the bitwise operators in C# I find myself using now.

OperatorUsageDescription
Bitwise AND a & b Returns a one in each bit position for which the corresponding bits of both a and b are ones.
Bitwise OR a | b Returns a one in each bit position for which the corresponding bits of a OR b or both a AND b are ones.
Bitwise XOR a ^ b Returns a one in each bit position for which the corresponding bits of either a OR b but not both a AND b are ones.
Bitwise NOT !a Inverts each bit of a.

For more on how Binary works, see http://www.swansontec.com/binary.html

As an aside - I had never really thought about whether zero is even or odd. Turns out zero is even, and here’s why http://en.wikipedia.org/wiki/Parity_of_zero

Tags: | Categories: C#

In my day job I have to deal with a lot of spreadsheet-type data stored as Comma-Separated Values.  One way to work with these files is in a text editor like Windows Notepad, or my favorite editor Notepad++.  However, sometime you want to have the power of Excel to do data analysis, formatting or sorting.

The problem is that when you "open" a file saved as a .csv with Excel you can run into data formatting problems as Excel tries to render the data in the most logical format. Here is an example.  I have a file of occupational data that looks like this:

Occupation,Occupation Title
"00-0000","Total, All Occupations"
"11-0000",Management Occupations
"11-1000",Top Executives
"11-1011",Chief Executives
"11-1021",General and Operations Managers
"11-1031",Legislators
"11-2000","Advertising, Marketing, and Sales Managers"
"11-2011",Advertising and Promotions Managers
"11-2021",Marketing Managers
"11-2022",Sales Managers
...

When I open the file in Excel, it mangles the occupation codes, trying to convert them to dates:

Occupation	Occupation Title
00-0000	Total, All Occupations
11-0000	Management Occupations
11-1000	Top Executives
11-1011	Chief Executives
11-1021	General and Operations Managers
11-1031	Legislators
Nov-00	Advertising, Marketing, and Sales Managers
Nov-11	Advertising and Promotions Managers
Nov-21	Marketing Managers
Nov-31	Sales Managers

The solution (in Excel 2010) is to:

  1. Open a blank workbook, then choose File > From Text.
  2. Choose your .csv file and click "Import".
  3. Choose the "Delimited" file type and click "Next".
  4. Enter the appropriate Delimiters until the Data Preview shows you the right column breaks and click "Next".
  5. Pick your 'problem' columns and change the "Column Data Format" to Text.
  6. Click "Finish".
Tags: | Categories: General

I was recently working through some old code that my team generated. One common pattern that I have run into is a 'check' for whether or not to send email. See, typically we store a configuration variable named something like "DeliverEmail" and set it to false when the code is on a development or beta machine. Then, in our code we read the config setting in an if() statement and ignore the send mail code if it is false.

This has worked for us for years, but it's a kludge. So I spent some time on the internets and found this article on configuring ASP.Net and SMTP to save outgoing email to a directory, rather than process and send it.

There are actually several reasons why you might want to write your email to a local file rather than send it immediately.
  • Testing - You may be using cleansed/obfuscated data and not want to send the messages. Or you may want to read text from the email to verify that it is formatted correctly, without having to manually open and check email sent to a personal account.
  • Hassle of configuring SMTP in Vista and Win 7 (see this article). SMTP isn't on or even installed by default in several MS operating systems.
There are two steps to setting this up: the code to send the message, which should be nearly identical to your normal SendMail code; and SMTP settings stored in your config file.
 
The code I use to send email is shown below.  There are two things to note. First, none of the SMTP server details are in this code; they are read from the config file at runtime.  Second, the MailMessage and SmtpClient objects are instantiated with the .Net Using{} syntax. I've occasionally had trouble with the SMTP cache failing to flush in a timely manner.  Both of these object implement IDispose, and the garbage collection process seems to flush the cache consistently.
public void SomeAction()
{
     using (MailMessage email = new MailMessage())
     {
          email.From = new MailAddress("no-reply@test.com");
          email.To.Add("test@test.com");
          email.Subject = “Test email to file”;
          email.Body = “This is body text saved to a .eml file”;
                
          using (SmtpClient client = new SmtpClient())
          {
               client.Send(email);
          }
     }
}
 The more recent schema for web.config includes a "mailSettings" node in the system.net section.  Within that section you can set the values as shown here to force the .Net SmtpClient to send the message as a file to a local directory:
<system.net>
    <mailSettings>
      <smtp deliveryMethod="SpecifiedPickupDirectory">
        <network host="localhost" />
        <specifiedPickupDirectory pickupDirectoryLocation="c:\deleteme\test_email\"/>
      </smtp>
    </mailSettings>
  </system.net>
Two final things to note.  For this to work the directory listed in pickupDirectoryLocation must already exist (the process won't create it if it is missing), and the account under which your app is running must have write/modify access to that directory,
Tags: | Categories:

February 1216

My How Time Flies

I spent the better part of my free time the last couple days upgrading this blog from BlogEngine.Net version 1.6 to version 2.5.  It wasn't a seamless or painless upgrade, but it didn't kill me either.  The last 18 months at work tried to do that...

To catch you up, between August 2010 and November 2011 the division/team I supervise was reorganized 3 times as projects and people were shuffled between supervisors.  Part of the reason for this was that we were trying to do a project originally planned for 3 years in about 14 months of active development.  We staffed up to 20 people, split the team, combined units and re-split again... it was a high level of chaos.

In the middle of all that, I fell in love and married the Maestra - best decision of my life! In the process I inherited a lovely college-age daughter and fantastic H.S. age son, to go along with my two boys.

Clearly, blogging consistency is not a strong-suit for me, but over the last 18 months a lot of technical and professional career lessons have been learned, and I hope to do a better job of sharing them with the world.

Tags: | Categories: General

(This post is from my old blogger site - thought it was time to move it here for safe keeping.)

Several months ago we began to transition from an in-house logging framework to Log4Net in our primary web applications. For the last couple of days I have been trying to troubleshoot some production problems in the two of these applications and I noticed that my Log4Net logs were using a 12hr clock for the timestamp, and didn't include an AM/PM indicator. It also used old style parameter names rather than the newer verbose parameters.

Seeing as how I tend to cut and paste code for things like that, and I know other programmers do as well, I figured I would blog about what I am switching to, as well as a couple of other options. In case you are not really familiar with log4net, this setting is changed by editing the value string for the ConversionPattern param in the - log4net --appender - section of the web or app config file:

<param name="ConversionPattern" value="your string here"/>


The old:
value="%-5p %d{yyyy-MM-dd hh:mm:ss} - %m%n"
Results in:
ERROR 2008-12-03 01:28:38 - login.aspx submitted for 7234

Besides the timestamp problem, the format string is difficult to understand. It is not at all obvious what 'p', 'd', 'm' and 'n' stand for. Well, maybe the fact that 'd' is followed by a date format string you could guess that it stood for Date.

The following link lists both the shortcut and the verbose fomat values from the Apache log4net documentation.

The one I am switching to:
value="%-5level %date{yyyy-MM-dd HH:mm:ss} - %message%newline">

Note the more obvious setting names. This value string results in:
INFO 2008-12-11 14:06:59 - login.aspx submitted for 7234


Other choices:
value="%-5level %date{G} - %message%newline">
Results in
INFO 12/11/2008 2:15:07 PM - login.aspx submitted for 7234

value="%-5level %date{yyyy-MM-dd hh:mm:ss t} - %message%newline">
Results in
INFO 2008-12-11 02:06:59 P - login.aspx submitted for 7234

value="%-5level %date{G} - %message%newline">
Results in
INFO 12/11/2008 2:15:07 PM - login.aspx submitted for 7234

value="%-5level %date{u} - %message%newline"
Results in (not this is UTC time)
INFO 2008-12-11 14:06:59Z - login.aspx submitted for 7234

Those of you familiar with date string formats in the .Net framework will recognize some of these strings. The Log4Net framework allows you to use any date format that is valid in a ToString() call in the .Net framework. For additional format strings see the MSDN Format String Documentation.

Tags: | Categories:

September 1003

Break on All Errors

Like a lot of .Net developers, I actually got into professional programming on the VB/VBA track.  I worked in Access and VB6 and took the grief from C++ developers, Java developers, FoxPro developers... you get the idea. 

As I transitioned to .Net I have found myself frequently switching between C# and VB.Net.  I love C# syntax, and I love the improvements in each version of Visual Sutdio, but there was one feature I really missed.  I don't know if it was the default in VB6, but you could set the IDE to break on all errors, even if they were handled. I never took the time to figure out if you could do that in VS 2005 or earlier, but I spent some time reviewing code of some of the developers that work for me this week, and I kept seeing a few weird bugs.  I set breakpoints and noticed that there were exceptions thrown, but they were swallowed in the Catch{} statement.

This lead to the discovery of the following VS 2008 Setting:

Debug > Exceptions (Ctrl D, E) >  Common Language Runtime Exceptions: check the "Thrown" box.

The behavior of the IDE seems identical to the old Break on All Errors from VB6.  In my case, it helped me track down errors in three different classes today.

 

Tags: | Categories:

I have been converting a number of test classes from NUnit to the Microsoft Test Framework (MSTest).  There are a number of quirky differences between the two, but the one that got me last night was how to use module-level variables.

Frequently in NUnit you (or at least I) use [TestFixtureSetup] to set module level variable (like connection strings) eg.

[TestFixture]	
public class AreaDAOTest
{
     private string _myVar;

     [TestFixtureSetUp]
     public void Init()
     {
         _myVar = //whatever here...;
     }
This doesn't work in MSUnit because [TestClassInitialize] is required to be a static and can't set reference variable.

The two options I tried were to use [TestInitialze] or a test class constructor. [TestInitialze] is run every test as expected, but what we didn't expect was that the constructor seemed to be called for each test as well.

The better pattern, discovered by a co-worker (thanks, Paul) for setting module level variables is:

[TestClass]
public class AreaDAOTest
{
     static string _myStaticVar;

     [ClassInitialize]
     public static void TestSetup(TestContext MSTestContext)
     {
          _myStaticVar = //whatever here...;
     }
Tags: | Categories:

On one of the primary web applications I maintain we have real-time help available to clients through an online chat application.  Usually when one of the chat line workers gets an interesting bug report they IM me and I try to fix the problem.

For weeks now I have occasionally been getting questions about our date validation. Specifically, we have people entering today's date and getting told that the date they have to enter must be today or earlier.  We have about 35 places where you can enter a date and the business requirement actually is that the date entered be 'today' or earlier.  We validate this on the server side, but also on the client using the JavaScript function show below. 

//Validate Past date
//---------------------------------------------//
function validatePastDate (control, sFieldName) 
{
    var c = document.getElementById(control);
    if (c != null ) 
    {
    //validate hasDate and isDate snipped//
        var d = new Date();
        var now = new Date();
        if(d.setTime(Date.parse(c.value, "n/j/y")) > now) 
        {
          sErr = sFieldName + " '" + c.value + "' must be a date earlier than today. \n"; 
        }
    }
    return sErr;    
}

Since the bug was opened in our issue database four different testers have tried to replicate it and none have been successful.  Yesterday I got pinged on IM and told that the 'funny date thing' was happening again.

I dropped everything and decided it was time to fix this once and for all.  I poured over the code, looking at the function and everywhere it was called.  No luck.  I ran it from my development box and from the beta server.  No dice.  I fired up an ancient desktop box that was kicking around and all of a sudden...ERROR!

Then I looked in the bottom right hand corner of the screen and noticed that it was 3:06 AM.  Bingo!  When I double-clicked the clock I discovered that the system date on that computer was 4/1/09, not 4/6/09.  It's clear now, but we all overlooked that

 

var now = new Date(); 

would report the client's system time to the client side validation...

Tags: | Categories:

I typically find myself using the Window Search all the time. It does a great job of finding files and documents by name, filtering the criteria by date or size, and I like having results in the graphical file explorer where I can sort by several attributes. If it could find my keys at 5:30 in the morning it would be the primary function on my laptop.

My recent experience with Search has been less satisfactory. I recently inherited several thousands of lines of HTML, asp, VB 6 and JavaScript code in hundreds of files that makes up a major portion of my employers web site. At the same time our database admins rolled out a new version of one of our primary databases that serves as a back end for several desktop and web applications. Unfortunately the web site wasn't checked thoroughly for dependency on the old database, nor compatibility with the new one. (That should be a topic for a whole separate stream of posts.)

Needless to say, this week has been one composed of frantic scrambling trying to figure out what was up and what was broken, and why. In a little bit of serendipity, I found that most of the applications used one particular configuration file in which was stored the connection information tagged with a label like 'ORA_CONNECT' (Nice job, previous developers.) When an application needed to connect it would call to the root config file and get the connection information by passing the label. Pretty standard stuff.

So, my first thought was to use the Windows Search to see if I could find all files that contained the string "ORA_CONNECT". So navigated to the root directory, fired up search, entered the parameters and waited with eager expectation after clicking on search. A long list of files came up, but shockingly it did not include the three files I had looked at by hand to determine what connection was being used.

Puzzled, I set up a little test. I created a directory with two subdirectories. In the root I put a text doc, and in the subdirectories I placed an HTML, ASP and MS Word file. Each file included text that contained the string ORA_CONNECT.

I ran search on that directory, and it found everything EXCEPT the use in the ASP page. I even went back and made sure that the All Files and Include Subdirectories options were enabled. Clearly Search was not reliable.

What will I FIND

I IM'd a co-worker about what I was tying to do, and he sent me a string using the FIND command. Ah Ha! Good old find! I had forgotten that since the last time I spent hours writing MS-DOS 5 batch files years ago. Oh No! Bad FIND...


As you can see from the image, access was denied to the subdirectories. FIND does not recurse. I knew I could write a batch to recurse the directories, and save the output to a file, etc, etc; but who has that kind of time?


FINDSTR /i /s ss64.com *.*

It was then that I remembered reading somewhere that there was a 'find' in windows that supported regular expressions like grep. I quick check of my links turned up the excellent site ss64.com - a command line reference site for SQL Server and Oracle, which just happens to have an XP/NT section.

The page on the FINDSTR command had just exactly what I needed:


Check out the command reference on ss64.com for other options, like printing the line and line number that match, and outputting the results to a text file.

This post was originally published on geekcyclist.blogspot.com on 10/5/2007

Tags: | Categories:

Welcome to the new AgileCoder.Net blog.

 Over the next few weeks I will be moving the technology related posts over here from my old blog at geekcyclist.blogspot.com.

 Please stay tuned!

Tags: | Categories: