Showing posts with label hack. Show all posts
Showing posts with label hack. Show all posts

Friday, January 22, 2016

Trick: How to mentally convert and calculate rate of pay


I have been real busy lately, so I will share only a short tip this week. However, I have some cool new projects/concepts I have been working on, such as a term rewriting system, so keep checking back.

I have been interviewing for a new job, its time to move up. Often I am quoted an annual salary, and I want to see how that compares with hourly wage. I do this calculation in my head, on the spot, and so can you. This technique leverages Estimation/Approximation.

Since 40hrs/week times 52wks/year = 40 * 52 = 2080 full-time work hours in a year.
We can use approximation by multiplying or dividing by 2000. Since we know that 2 * 1000 = 2000, multiplying/diving by 2000 is trivial. Remember, to multiply or divide by any power of ten, now matter how great, just count the zeros and just shift the decimal place once to the right, or towards a smaller quantity, that number of times. e.g. 43.50 * 1000 = 43,500





'So how large is the error from estimating?', One might ponder... Well ponder no more! Hark:


Estimation Error - From hourly to yearly--

$12/HR @ 40HR/WK
 
  $24,000/YR - Estimated
  $24,960/YR - Actual
  -------
     -$960  - Difference

$25/HR @ 40HR/WK
 
  $50,000/YR - Estimated
  $52,000/YR - Actual
  -------
   $2,000/YR - Difference

$50/HR @ 40HR/WK
 
 $100,000/YR - Estimated
 $104,000/YR - Actual
  --------
   $4,000  - Difference
 
 
Estimation Error - From yearly to hourly--
 
 $25K/YR @ 40HR/WK
  
   $12.50/HR - Estimated
   $12.02/HR - Actual
   ------
  -$0.48/HR  - Difference
 
 $50K/YR @ 40HR/WK
  
   $25.00/HR - Estimated
   $24.04/HR - Actual
   ------
   -$0.96/HR
     
 $100K/YR @ 40HR/WK
  
   $50.00/HR - Estimated
   $48.07/HR - Actual
   ------
   -$1.92/HR







Sunday, August 2, 2015

Certificate Enumerator



     Recently, my windows quit updating. Just prior to that, I had been messing around with my certificate store, so I suspected that to be the cause. Running Microsoft's troubleshooter reset the download, which I had a lot of hope of fixing the issue, but the download still continued to to fail. I decided to check the Windows Event Logs, and that's where I found an error message about a certificate in the chain failing. I knew it! However, I did not know whether a trusted certificate accidentally got put in the untrusted store, or whether an untrusted certificate was accidentally put in the trusted store. I needed a way to search all of my certificates' thumbprints or serial numbers against a know repository of trusted or untrusted certificates.
    Microsoft's certificate snap-in for MMC does not allow you to view certificates in an efficient way. Opening them one at a time, manually, and then scrolling all the way down to where the thumbprint is displayed to compare it to a webpage is painful. Also, I was not satisfied with the way that Microsoft allows you to search the certificate store. The search is not very effective and you cant even search for thumbprints! Also I do not believe the search feature allows you anyway to copy any of that information to clipboard.
Most of what I needed to accomplish could simply be done if I could just export all of my computers certificates thumbprint or serial numbers to a text file, csv file, or other simple and searchable format. Then I thought to myself, I know how to do that! It was the great the lack of features of the MMC certificate snap-in, and the inability to search for certificate thumbprints that inspired me to write my own certificate utility, known simply as Certificate Enumerator.




     CertificateEnumerator can list every certificate in your various certificate stores for your local machine and currently logged in user. It can then display that information to you either in a DataGridView or TextBox (as columnarized text), and provides the ability to persist that information to file as text, comma separated values (CSV), excel format or HTML table.


     The Certificate Enumerator also has the ability to 'validate' each certificate against its CRL (certificate revocation list), if it supplied one.


     The GUI could really use some love. In case you missed it, the project is on my GitHub, so feel free to download the source and play with it. If you come up with useful, submit a pull request.


Friday, April 24, 2015

Install C# on Raspberry Pi 2 with Mono



Intro

We can run C# on the Raspberry Pi 2! This has made the raspberry pi very valuable in my eyes. You can use Visual Studio to compile a .exe and then RUN IT on The Pi using Mono.

What I liked about The Pi: Getting it working was dead simple. I just formatted an SD card in FAT 32, downloading and extracting a 500mb .zip file and copying the contents of the extracted folder onto the SD card. After you've accomplished that, go take a beer break, you deserved it *whew*.

I was able to just plug in the cables, put in the SD card and I running Linux on The Pi. I was using the HDMI and Ethernet ports, however. If you wish to use the tiny, headphone-jack-looking component video or a USB wireless adapter however, there will be additional steps.


Problems / Gotchas 

Winforms may not play nice with Mono when on the Raspberry pi. Specifically, the problem manifests when you attempt to use a TextBox on your form, but who uses those? This bug  is supposedly fixed in the latest releases, but I have yet to get it to work by just updating my mono. It is very likely that I have to REBUILD mono from the latest source on the pi, which can take several hours. I have not tested this. The bug tracker for this says it has to do with with the pi's 'hard floats', which is referring to hardware floating point calculations.



The hardware is not exactly what I would call stable. Sometimes it does not POST (the BIOS is a binary blob, so does a pi really have a POST?). Anyways, ensure your USB charger that plugs into the wall that can supply plenty of power. The websites say 700ma at a minimum. I find using a 700ma charger is not sufficient. I use a charger that can push 2A, and I dont see many problems.




How does this work? 

A project called Mono. The Mono team has implemented a Common Language Runtime (CLR) that runs in the Linux and Mac environments and is coded to the ECMA-335 Common Language Infrastructure (CLI) AND covers functions and classes in the .NET framework, going all the way to 4.0 with some 4.5! Its open source, of course, and also has a compiler that follows the ECMA-334 C# Language Specification to turn C# code into CLI intermediate code that can be ran in windows as well. Remember, the CLI is just a specification that any language could (theoretically) be compiled to CLI.




Installing Mono C# on Raspberry Pi

This step was pretty straight forward as well. The instructions below are if you installed the NOOBS package which uses Raspian. If you have a different flavor of Linux, the commands below might be different.

You should already have sudo installed on your raspb pi. Prefixing a command with sudo allows you to run processes under a higher privilege without having to log into root. Also if you didnt know, the default username/password to log in the first time is pi/raspberry. If you prefer, you can type 'startx' to bring up the xwindows GUI. The commands below need to be entered into a terminal, which can be brought up within the GUI.

Fist, you want to update your Raspian to the latest version. To do that, you have to have the raspberry pi hooked up the the internet. If you have an ethernet cable and the router uses DHCP, plugging in the ethernet will be sufficient. Then, at the console simply enter the below line into the console:

sudo apt-get update

That may take a few minutes to complete and might prompt you to hit Y/N regarding the size of the package.

sudo apt-get install mono-runtime

This will also will take a few minutes to complete and might prompt you to hit Y/N regarding the size of the package.

After you've accomplished that, go have another beer because that was hard work!




Mono can now be used to run .NET executables:

mono Test.exe

Or, if your program requires elevated rights:

sudo mono Test.exe


Mono also features a REPL (read eval print loop). To access it, type 'csharp'. You can use your favorite text editor to make .cs files and compile them. My recommendation is, of course, to use Microsoft's Visual Studio. You can build and compile an application with MSVS in windows, and just transfer the executable by SCP or thumb-drive to the raspberry pi, which is pretty slick. Hats off to the developer(s) of Mono.


Tuesday, October 28, 2014

Create C# Class Code From a DataTable using CodeDOM



Note by author:

   Since writing this, I have expanded on this idea quite a bit. I have written a lightweight ORM class library that I call EntityJustWorks.

   The full project can be found on
GitHub or CodePlex.


   EntityJustWorks not only goes from a class to DataTable (below), but also provides:

Security Warning:
This library generates dynamic SQL, and has functions that generate SQL and then immediately executes it. While it its true that all strings funnel through the function Helper.EscapeSingleQuotes, this can be defeated in various ways and only parameterized SQL should be considered SAFE. If you have no need for them, I recommend stripping semicolons ; and dashes --. Also there are some Unicode characters that can be interpreted as a single quote or may be converted to one when changing encodings. Additionally, there are Unicode characters that can crash .NET code, but mainly controls (think TextBox). You almost certainly should impose a white list: string clean = new string(dirty.Where(c => "abcdefghijklmnopqrstuvwxyz0123456789.,\"_ !@".Contains(c)).ToArray()); 
PLEASE USE the SQLScript.StoredProcedure and DatabaseQuery.StoredProcedure classes to generate SQL for you, as the scripts it produces is parameterized. All of the functions can be altered to generate parameterized instead of sanitized scripts. Ever since people have started using this, I have been maintaining backwards compatibility. However, I may break this in the future, as I do not wish to teach one who is learning dangerous/bad habits. This project is a few years old, and its already showing its age. What is probably needed here is a total re-write, deprecating this version while keep it available for legacy users after slapping big warnings all over the place. This project was designed to generate the SQL scripts for standing up a database for a project, using only MY input as data. This project was never designed to process a USER'S input.! Even if the data isn't coming from an adversary, client/user/manually entered data is notoriously inconsistent. Please do not use this code on any input that did not come from you, without first implementing parameterization. Again, please see the SQLScript.StoredProcedure class for inspiration on how to do that.


So far I have posted several times on the DataTable class. I have shown how to convert a DataTable to CSV or tab-delimited file using the clipboard, how to create a DataTable from a class using reflection, as well as how to populate the public properties of a class from a DataTable using reflection. Continuing along these lines, I decided to bring the DataTable-To-Class wagons around full-circle and introduce a class that will generate the C# code for the class that is used by the DataTableToClass<T> function, so you don't have to create it manually. The only parameter required to generate the C# class code is, of course, a DataTable.

The code below is rather trivial. It uses CodeDOM to build up a class with public properties that match the names and data types of the data columns of the supplied DataTable. I really wanted the output code to use auto properties. This is not supported by CodeDOM, however, so I used a little hack or workaround to accomplish the same thing. I simply added the getter and setter code for the property to the member's field name. CodeDOM adds a semicolon to the end of the CodeMemberField statement, which would cause the code not to compile, so I added two trailing slashes "//" to the end of the field name to comment out the semicolon. The whole point of creating auto properties was to have clean, succinct code, so after I generate the source code file, I clean up the commented-out semicolons by replacing every occurrence with an empty string. The main disadvantage of this 'workaround' is that the code cannot be used to generate a working class in Visual Basic code. I do have proper CodeDOM code that does not employ this workaround, but I prefer the output code to contain auto-properties; auto-generated code is notorious for being messy and hard to read, and I did not want my generated code to feel like generated code.


Below is the DataTableToCode function, its containing class and its supporting functions. The code is short, encapsulated, clean and commented, so I will just let it speak for itself:

public static class DataTableExtensions
{
   public static string DataTableToCode(DataTable Table)
   {
      string className = Table.TableName;
      if(string.IsNullOrWhiteSpace(className))
      {   // Default name
         className = "Unnamed";
      }
      className += "TableAsClass";
      
      // Create the class
      CodeTypeDeclaration codeClass = CreateClass(className);
      
      // Add public properties
      foreach(DataColumn column in Table.Columns)
      {
         codeClass.Members.Add( CreateProperty(column.ColumnName, column.DataType) );
      }
      
      // Add Class to Namespace
      string namespaceName = "AutoGeneratedDomainModels";
      CodeNamespace codeNamespace = new CodeNamespace(namespaceName);
      codeNamespace.Types.Add(codeClass);
      
      // Generate code
      string filename = string.Format("{0}.{1}.cs",namespaceName,className);
      CreateCodeFile(filename, codeNamespace);
      
      // Return filename
      return filename;
   }
   
   static CodeTypeDeclaration CreateClass(string name)
   {
      CodeTypeDeclaration result = new CodeTypeDeclaration(name);
      result.Attributes = MemberAttributes.Public;
      result.Members.Add(CreateConstructor(name)); // Add class constructor
      return result;
   }
   
   static CodeConstructor CreateConstructor(string className)
   {
      CodeConstructor result = new CodeConstructor();
      result.Attributes = MemberAttributes.Public;
      result.Name = className;
      return result;
   }
   
   static CodeMemberField CreateProperty(string name, Type type)
   {
      // This is a little hack. Since you cant create auto properties in CodeDOM,
      //  we make the getter and setter part of the member name.
      // This leaves behind a trailing semicolon that we comment out.
      //  Later, we remove the commented out semicolons.
      string memberName = name + "\t{ get; set; }//";
      
      CodeMemberField result = new CodeMemberField(type,memberName);
      result.Attributes = MemberAttributes.Public | MemberAttributes.Final;
      return result;
   }
   
   static void CreateCodeFile(string filename, CodeNamespace codeNamespace)
   {
      // CodeGeneratorOptions so the output is clean and easy to read
      CodeGeneratorOptions codeOptions = new CodeGeneratorOptions();
      codeOptions.BlankLinesBetweenMembers = false;
      codeOptions.VerbatimOrder = true;
      codeOptions.BracingStyle = "C";
      codeOptions.IndentString = "\t";
      
      // Create the code file
      using(TextWriter textWriter = new StreamWriter(filename))
      {
         CSharpCodeProvider codeProvider = new CSharpCodeProvider();
         codeProvider.GenerateCodeFromNamespace(codeNamespace, textWriter, codeOptions);
      }
      
      // Correct our little auto-property 'hack'
      File.WriteAllText(filename, File.ReadAllText(filename).Replace("//;", ""));
   }
}


An example of the resulting code appears below:

namespace AutoGeneratedDomainModels
{
   public class CustomerTableAsClass
   {
      public CustomerTableAsClass()
      {
      }
      public string FirstName   { get; set; }
      public string LastName    { get; set; }
      public int  Age           { get; set; }
      public char Sex           { get; set; }
      public string Address     { get; set; }
      public string Birthdate   { get; set; }
   }
}

I am satisfied with the results and look of the code. The DataTableToCode() function can be a huge time saver if you have a large number of tables you need to write classes for, or if each DataTable contains a large number of columns.

If you found any of this helpful, or have any comments or suggestions, please feel free to post a comment.

Saturday, September 27, 2014

Resize form to match the contents of DataGridView



Sometimes the sole purpose of a Form is to display a DataGridView. In that case, you probably want the Form to automatically resize to the size of the contents in the DataGridView. I've seen solutions that loop through all the rows and add up the height, but this is ugly, and usually does not take into account margins, padding, DividerHeight and row header padding. There must be a better way...

My strategy is to temporarily undock the DataGridView, set AutoSize to true, then capture the DataGridView's Size at that point, then restore the Dock and AutoSize property. Then use the captured size to resize the Winform:


// Within the Form class
private void AutoSizeFormToDataGridView()
{
 Size contentsSize = GetDataGridViewContentsSize();
 this.ClientSize = contentsSize;
}

protected Size GetDataGridViewContentsSize()
{
 DockStyle dockStyleSave = dataGridView1.Dock;
 dataGridView1.Dock = DockStyle.None;
 dataGridView1.AutoSize = true;
 
 Size dataContentsSize = dataGridView1.Size;
 
 dataGridView1.AutoSize = false;
 dataGridView1.Dock = dockStyleSave;
 return dataContentsSize;
}


Or alternatively you can define this as an extension method:

public static Size GetContentsSize(this DataGridView dataGrid) { //...


Enjoy!

Friday, September 26, 2014

DataTable or DataGridView to CSV or HTML file using Clipboard



It turns out that DataGridView.GetClipboardContent() returns all the selected cells of a DataGridView as a type DataObject, which is conveniently consumed by the Windows.Forms.Clipboard class, as well as other WYSIWYG editors from Microsoft. From this you can set the Clipboard, then get the clipboard various data formats, including:
- Comma separated value
- Tab separated value
- HTML

So instead of looping though columns and then rows, you can output the entire DataGridView as a CSV file in just 3 lines of code! (4 if you count setting the ClipboardCopyMode, which can be set in the Form Builder.

Here is the code:

void DataGridViewToCSV(string Filename)
{
   bool allowAddRows = dataGridView1.AllowUserToAddRows;
   bool rowHeadersVisible = dataGridView1.RowHeadersVisible;
   dataGridView1.AllowUserToAddRows = false;
   dataGridView1.RowHeadersVisible = false;

   // Choose whether to write header. You will want to do this for a CSV file.
   dataGridView1.ClipboardCopyMode = DataGridViewClipboardCopyMode.EnableAlwaysIncludeHeaderText;
   // Select the cells we want to serialize.
   dataGridView1.SelectAll(); // One could also use DataGridView.Rows[RowIndex].Selected = true;

   // Save the current state of the clipboard so we can restore it after we are done
   IDataObject objectSave = Clipboard.GetDataObject();
   // Copy (set clipboard)
   Clipboard.SetDataObject(dataGridView1.GetClipboardContent());
   // Paste (get the clipboard and serialize it to a file)
   File.WriteAllText(Filename,Clipboard.GetText(TextDataFormat.CommaSeparatedValue));
   // Restore the current state of the clipboard so the effect is seamless
   if(objectSave != null)
   {
      Clipboard.SetDataObject(objectSave);
   }
   dataGridView1.AllowUserToAddRows = allowAddRows;
   dataGridView1.RowHeadersVisible = rowHeadersVisible;
}

Some improvements

For a tab-delimited file, use the TextDataFormat.Text enum in your call to Clipboard.GetText(). You can also output your DataGridView as HTML by using TextDataFormat.Html instead of TextDataFormat.CommaSeparatedValue, but there is extra header data you have to parse out:

   string result = Clipboard.GetText(TextDataFormat.CommaSeparatedValue);
   result = result.Substring( result.IndexOf("") );

Notes:
- An object must be serializable for it to be put on the Clipboard.

Sunday, August 3, 2014

Set Public Properties of C# class from a DataTable using reflection



Note by author:

   Since writing this, I have expanded on this idea quite a bit. I have written a lightweight ORM class library that I call EntityJustWorks.

   The full project can be found on
GitHub or CodePlex.


   EntityJustWorks not only goes from a class to DataTable (below), but also provides:



Security Warning:
This library generates dynamic SQL, and has functions that generate SQL and then immediately executes it. While it its true that all strings funnel through the function Helper.EscapeSingleQuotes, this can be defeated in various ways and only parameterized SQL should be considered SAFE. If you have no need for them, I recommend stripping semicolons ; and dashes --. Also there are some Unicode characters that can be interpreted as a single quote or may be converted to one when changing encodings. Additionally, there are Unicode characters that can crash .NET code, but mainly controls (think TextBox). You almost certainly should impose a white list:
string clean = new string(dirty.Where(c => "abcdefghijklmnopqrstuvwxyz0123456789.,\"_ !@".Contains(c)).ToArray());

PLEASE USE the SQLScript.StoredProcedure and DatabaseQuery.StoredProcedure classes to generate SQL for you, as the scripts it produces is parameterized. All of the functions can be altered to generate parameterized instead of sanitized scripts. Ever since people have started using this, I have been maintaining backwards compatibility. However, I may break this in the future, as I do not wish to teach one who is learning dangerous/bad habits. This project is a few years old, and its already showing its age. What is probably needed here is a total re-write, deprecating this version while keep it available for legacy users after slapping big warnings all over the place. This project was designed to generate the SQL scripts for standing up a database for a project, using only MY input as data. This project was never designed to process a USER'S input.! Even if the data isn't coming from an adversary, client/user/manually entered data is notoriously inconsistent. Please do not use this code on any input that did not come from you, without first implementing parameterization. Again, please see the SQLScript.StoredProcedure class for inspiration on how to do that.




In this post I showed how to create a DataTable where the column names and types matched the properties of a class. In this post, we work the opposite direction and start with a Data-First approach. Given an SQL Database, we can easily convert a query to a DataTable using System.Data's SqlDataAdapter.Fill method. Now, given a DataTable, I show you here how to use Reflection to populate a class's public properties from a DataRow in a DataTable (or a List<> of classes, one from each DataRow in the DataTable) where the ColumnName matches the name of the public property in the class exactly (case-sensitive).
If the DataTable has extra columns that don't match up to a property in the class, they are ignored. If the DataTable is missing columns to match a class property, that property is ignored and left at the default value for that type (since it is a property). If you desire the ColumnName/PropertyInfo.Name matching behavior to be case insensitive, simply modify the line that compares the two strings (PropertyInfo.Name and DataColumn.ColumnName) to include a call to String.ToUpper() or String.ToLower() for each name.

If you paying close attention, or have ever attempted this kind of thing before, you are probably thinking to yourself that the most laborious (and error-prone) process is going to be creating the C# classes plus their many auto-properties that have to match the columns of a table, all manually. Well, take solace in the fact that I already thought of this and created a solution to generate C# class object code files from a DataTable using CodeDOM. It even implements a little hack to generate the properties as auto-properties (something not supported by CodeDOM) for clean, compact code that isn't bloated with private backing fields, and full getter/setter implementation.
Ultimately, the goal is to have a full, end-to-end, class-to-DataTable-to-SQL and back-again class library solution. Something like a poor-man's Entity Framework, or minimum-viable ORM. So stay alert for the next piece that will bring these wagons 'round full-circle: Automatic generation of SQL CREATE, INSERT INTO, and UPDATE scripts from a DataTable, which was generated from a C# class object, which can be generated from a DataTable, which can be generated by a SQL Database, which can be... well you get the idea.

This code has been tested and is a a little more robust than some of the equivalent samples I have been finding on StackOverflow (such as being able to handle properties of type Nullable<>. However there probably exists some conditions or use cases that I have not thought of, so please feel free to leave a comment if you find a way I can improve this class or have a feature request. In the next paragraph, I describe what the code is doing, or if you don't care, you can jump straight to the code below it. Enjoy.

How it works: Fist we get a list of PropertyInfo from the class. This will effectively be a list of properties in that class that we will want to fill. PropertyInfo exposes the Name property and the SetValue method, which takes an object and a value as parameters.
    We are going to make three nested loops to do this (one for each DataRow, one for each PropertyInfo and one for each DataColumn) and return a List of classes, each one filled out from a single row in the DataTable. It is possible to fill out one class provided a DataTable and row index in only two nested loops, and this post will provide that example too.
    For each row in DataTable.Rows, we will need to loop through each property (to fill them) and then loop through each DataTable's DataColumn and match the PropertyInfo.Name to the DataColumn.ColumnName. We then call the PropertyInfo's SetValue method. This function will take advantage of generics so that we can pass in any class as a parameter.

Here is the code:

public static class Helper
{
   public static class Table
   {
      /// <summary>
      /// Fills the public properties of a class from the first row of a DataTable
      ///  where the name of the property matches the column name from that DataTable.
      /// </summary>
      /// <param name="Table">A DataTable that contains the data.</param>
      /// <returns>A class of type T with its public properties matching column names
      ///      set to the values from the first row in the DataTable.</returns>
      public static T ToClass<T>(DataTable Table) where T : class, new()
      {
          T result = new T();
          if (Validate(Table))
          {  // Because reflection is slow, we will only pass the first row of the DataTable
              result = FillProperties<T>(Table.Rows[0]);
          }
          return result;
      }
       
      /// <summary>
      /// Fills the public properties of a class from each row of a DataTable where the name of
      /// the property matches the column name in the DataTable, returning a List of T.
      /// </summary>
      /// <param name="Table">A DataTable that contains the data.</param>
      /// <returns>A List class T with each class's public properties matching column names
      ///      set to the values of a diffrent row in the DataTable.</returns>
      public static List<T> ToClassList<T>(DataTable Table) where T: class, new()
      {
          List<T> result = new List<T>();
          
          if (Validate(Table))
          {
              foreach(DataRow row in Table.Rows)
              {
                   result.Add(FillProperties<T>(row));
              }
          }
          return result;
      }
       
      /// <summary>
      /// Fills the public properties of a class from a DataRow where the name
      /// of the property matches a column name from that DataRow.
      /// </summary>
      /// <param name="Row">A DataRow that contains the data.</param>
      /// <returns>A class of type T with its public properties set to the
      ///      data from the matching columns in the DataRow.</returns>
      public static T FillProperties<T>(DataRow Row) where T: class, new()
      {
          T result = new T();
          Type classType = typeof(T);
          
          // Defensive programming, make sure there are properties to set,
          //   and columns to set from and values to set from.
          if(    Row.Table.Columns.Count < 1
              || classType.GetProperties().Length < 1
              || Row.ItemArray.Length < 1)
          {
              return result;
          }
          
          foreach (PropertyInfo property in classType.GetProperties())
          {
              foreach(DataColumn column in Row.Table.Columns)
              {
                  // Skip if Property name and ColumnName do not match
                  if(property.Name != column.ColumnName)
                      continue;
                  // This would throw if we tried to convert it below
                  if(Row[column] == DBNull.Value)
                      continue;
                  
                  object newValue;
                  
                  // If type is of type System.Nullable, do not attempt to convert the value
                  if (IsNullable(property.PropertyType))
                  {
                      newValue = Row[property.Name];
                  }
                  else
                  {   // Convert row object to type of property
                      newValue = Convert.ChangeType(Row[column], property.PropertyType);
                  }
                  
                  // This is what sets the class properties of the class
                  property.SetValue(result, newValue, null);
              }
          }
          return result;
      }
       
      /// <summary>
      /// Checks a DataTable for empty rows, columns or null.
      /// </summary>
      /// <param name="DataTable">The DataTable to check.</param>
      /// <returns>True if DataTable has data, false if empty or null.</returns>
      public static bool Validate(DataTable DataTable)
      {
          if (DataTable == null) return false;
          if (DataTable.Rows.Count == 0) return false;
          if (DataTable.Columns.Count == 0) return false;
          return true;
      }
       
      /// <summary>
      /// Checks if type is nullable, Nullable<T> or its reference is nullable.
      /// </summary>
      /// <param name="type">Type to check for nullable.</param>
      /// <returns>True if type is nullable, false if it is not.</returns>
      public static bool IsNullable(Type type)
      {
          if (!type.IsValueType) return true; // ref-type
          if (Nullable.GetUnderlyingType(type) != null) return true; // Nullable<T>
          return false; // value-type
      }
   }
}

Tuesday, July 9, 2013

Procedural generation of cave-like maps for rogue-like games



This post is about procedural content generation of cave-like dungeons/maps for rogue-like games using what is known as the Cellular Automata method.

To understand what I mean by cellular automata method, imagine Conway's Game of Life. Many algorithms use what is called the '4-5 method', which means a tile will become a wall if it is a wall and 4 or more of its nine neighbors are walls, or if it is not a wall and 5 or more neighbors are walls. I start by filling the map randomly with walls or space, then visit each x/y position iteratively and apply the 4-5 rule. Usually this is preceded with 'seeding' the map by randomly filling each cell of the map with a wall or space, based on some weight (say, 40% of the time it chooses to place a wall). Then the automata step is applied multiple times over the entire map, precipitating walls and subsequently smoothing them. About 3 rounds is all that is required, with about 4-5 rounds being pretty typical amongst most implementations. Perhaps a picture of the the output will help you understand what I mean.

Using the automata-based method for procedural generation of levels will produce something similar to this:

Sure the dungeon generation by linking rooms approach has its place, but I really like the 'natural' look to the automata inspired method. I first originally discovered this technique on the website called Roguebasin. It is a great resource for information concerning the different problems involved with programming a rogue-like game, such as Nethack or Angband.

One of the major problems developers run into while employing this technique is the formation of isolated caves. Instead of one big cave-like room, you may get section of the map that is inaccessible without digging through walls. Isolated caves can trap key items or (worse) stairs leading to the next level, preventing further progress in the game. I have seen many different approaches proposed to solve this problem. Some suggestions I've seen include: 1) Discarding maps that have isolated caves, filling in the isolated sections, or finely tweaking the variables/rules to reduce occurrences of such maps. None of these are ideal (in my mind), and most require a way to detect isolated sections, which is another non-trivial problem in itself.

Despite this, I consider the problem solved because I have discovered a solution that is dead simple and almost** never fail because of the rules of the automata generation itself dictate such. I call my method 'Horizontal Blanking' because you can probably guess how it works now just from hearing the name. This step comes after the random filling of the map (initialization), but before the cellular automata iterations. After the map is 'seeded' with a random fill of walls, a horizontal strip in the middle of of the map is cleared of all walls. The horizontal strip is about 3 or 4 block tall (depending on rules). Clearing a horizontal strip of sufficient width will prevent a continuous vertical wall from being created and forming isolated caves in your maps. After horizontal blanking, you can begin applying the cellular automata method to your map.

** I say 'almost' because although it it not possible to get whole rooms that are disconnected from each other, it is possible to get tiny squares of blank space in the northern or southern walls that consist of about 4-5 blocks in total area. Often, these little holes will resolve themselves during the rounds of automata rules, but there still exists the possibility that one may persist. My answer to this edge case would be to use some rules around the placement of stairwells (and other must-find items) dictating that such objects must have a 2-3 block radius clear of walls to be placed.


See below for the code that produced the above screenshot, or click here to download the entire project with source code that you can compile yourself.

public class MapHandler
{
 Random rand = new Random();
 
 public int[,] Map;
 
 public int MapWidth   { get; set; }
 public int MapHeight  { get; set; }
 public int PercentAreWalls { get; set; }
 
 public MapHandler()
 {
  MapWidth = 40;
  MapHeight = 21;
  PercentAreWalls = 40;
  
  RandomFillMap();
 }

 public void MakeCaverns()
 {
  // By initilizing column in the outter loop, its only created ONCE
  for(int column=0, row=0; row <= MapHeight-1; row++)
  {
   for(column = 0; column <= MapWidth-1; column++)
   {
    Map[column,row] = PlaceWallLogic(column,row);
   }
  }
 }
 
 public int PlaceWallLogic(int x,int y)
 {
  int numWalls = GetAdjacentWalls(x,y,1,1);

  
  if(Map[x,y]==1)
  {
   if( numWalls >= 4 )
   {
    return 1;
   }
   return 0;   
  }
  else
  {
   if(numWalls>=5)
   {
    return 1;
   }
  }
  return 0;
 }
 
 public int GetAdjacentWalls(int x,int y,int scopeX,int scopeY)
 {
  int startX = x - scopeX;
  int startY = y - scopeY;
  int endX = x + scopeX;
  int endY = y + scopeY;
  
  int iX = startX;
  int iY = startY;
  
  int wallCounter = 0;
  
  for(iY = startY; iY <= endY; iY++) {
   for(iX = startX; iX <= endX; iX++)
   {
    if(!(iX==x && iY==y))
    {
     if(IsWall(iX,iY))
     {
      wallCounter += 1;
     }
    }
   }
  }
  return wallCounter;
 }
 
 bool IsWall(int x,int y)
 {
  // Consider out-of-bound a wall
  if( IsOutOfBounds(x,y) )
  {
   return true;
  }
  
  if( Map[x,y]==1  )
  {
   return true;
  }
  
  if( Map[x,y]==0  )
  {
   return false;
  }
  return false;
 }
 
 bool IsOutOfBounds(int x, int y)
 {
  if( x<0 data-blogger-escaped-else="" data-blogger-escaped-if="" data-blogger-escaped-return="" data-blogger-escaped-true="" data-blogger-escaped-x="" data-blogger-escaped-y="">MapWidth-1 || y>MapHeight-1 )
  {
   return true;
  }
  return false;
 }
Above is the main core of the logic.


Here is the rest of the program, such as filling, printing and blanking:
 
 public void PrintMap()
 {
  Console.Clear();
  Console.Write(MapToString());
 }
 
 string MapToString()
 {
  string returnString = string.Join(" ", // Seperator between each element
                                    "Width:",
                                    MapWidth.ToString(),
                                    "\tHeight:",
                                    MapHeight.ToString(),
                                    "\t% Walls:",
                                    PercentAreWalls.ToString(),
                                    Environment.NewLine
                                   );
  
  List<string> mapSymbols = new List();
  mapSymbols.Add(".");
  mapSymbols.Add("#");
  mapSymbols.Add("+");
  
  for(int column=0,row=0; row < MapHeight; row++ ) {
   for( column = 0; column < MapWidth; column++ )
   {
    returnString += mapSymbols[Map[column,row]];
   }
   returnString += Environment.NewLine;
  }
  return returnString;
 }
 
 public void BlankMap()
 {
  for(int column=0,row=0; row < MapHeight; row++) {
   for(column = 0; column < MapWidth; column++) {
    Map[column,row] = 0;
   }
  }
 }
 
 public void RandomFillMap()
 {
      // New, empty map
      Map = new int[MapWidth,MapHeight];
  
      int mapMiddle = 0; // Temp variable
      for(int column=0,row=0; row < MapHeight; row++) {
         for(column = 0; column < MapWidth; column++)
         {
       // If coordinants lie on the edge of the map
       // (creates a border)
       if(column == 0)
       {
      Map[column,row] = 1;
       }
       else if (row == 0)
       {
      Map[column,row] = 1;
       }
       else if (column == MapWidth-1)
       {
      Map[column,row] = 1;
       }
       else if (row == MapHeight-1)
       {
      Map[column,row] = 1;
       }
       // Else, fill with a wall a random percent of the time
       else
       {
      mapMiddle = (MapHeight / 2);
    
      if(row == mapMiddle)
      {
     Map[column,row] = 0;
      }
      else
      {
     Map[column,row] = RandomPercent(PercentAreWalls);
      }
       }
   }
      }
 }

 int RandomPercent(int percent)
 {
  if(percent>=rand.Next(1,101))
  {
   return 1;
  }
  return 0;
 }
 
 public MapHandler(int mapWidth, int mapHeight, int[,] map, int percentWalls=40)
 {
  this.MapWidth = mapWidth;
  this.MapHeight = mapHeight;
  this.PercentAreWalls = percentWalls;
  this.Map = new int[this.MapWidth,this.MapHeight];
  this.Map = map;
 }
}


And of course, the main function:
 
public static void Main(string[] args)
{
 char key  = new Char();
 MapHandler Map = new MapHandler();
 
 string instructions =
  "[Q]uit [N]ew [+][-]Percent walls [R]andom [B]lank" + Environment.NewLine +
  "Press any other key to smooth/step";

 Map.MakeCaverns();
 Map.PrintMap();
 Console.WriteLine(instructions);
 
 key = Char.ToUpper(Console.ReadKey(true).KeyChar);
 while(!key.Equals('Q'))
 {
  if(key.Equals('+')) {
   Map.PercentAreWalls+=1;
   Map.RandomFillMap();
   Map.MakeCaverns();
   Map.PrintMap();
  } else if(key.Equals('-')) {
   Map.PercentAreWalls-=1;
   Map.RandomFillMap();
   Map.MakeCaverns();
   Map.PrintMap();
  } else if(key.Equals('R')) {
   Map.RandomFillMap();
   Map.PrintMap();
  } else if(key.Equals('N')) {
   Map.RandomFillMap();
   Map.MakeCaverns();
   Map.PrintMap();
  } else if(key.Equals('B')) {
   Map.BlankMap();
   Map.PrintMap();
  } else if(key.Equals('D')) {
   // I set a breakpoint here...
  } else {
   Map.MakeCaverns();
   Map.PrintMap();
  }
  Console.WriteLine(instructions);
  key = Char.ToUpper(Console.ReadKey(true).KeyChar);
 }
 Console.Clear();
 Console.Write(" Thank you for playing!");
 Console.ReadKey(true);
}


See also: Roguebasin - Cellular Automata Method for Generating Random Cave-Like Levels


Thursday, June 27, 2013

Fake/Random Identity Generator




Inspiration


During my research on RSA cryptography and the importance of a truly random number for having a large key-space, I stumbled on to FakeNameGenerator.com. I thought the concept could be really useful for certain applications and could easily envision how to implement it in C#, and make it extensible/customizable.

Take a look at these:

<?xml version="1.0" standalone="yes"?>
<DocumentElement>
  <Order>
    <Date>3/18/2005</Date>
    <TrackingNumber>1Z 8A8 238 01 9398 182 1</TrackingNumber>
    <FirstName>Keaton </FirstName>
    <LastName>Day</LastName>
    <StreetAddress>4828 Cherry St.</StreetAddress>
    <City>Nanticoke</City>
    <State>SC</State>
    <Zip>89130</Zip>
    <Email>HaleHale8026@mail.com</Email>
    <Phone>425-765-4520</Phone>
  </Order>

  <Payroll>
    <PhoneNumber>971-258-5703</PhoneNumber>
    <AltPhoneNumber>501-769-1331</AltPhoneNumber>
    <FirstName>Xyla </FirstName>
    <LastName>Hoover</LastName>
    <EmployeeID>499</EmployeeID>
    <HireDate>5/28/2011</HireDate>
    <Birthdate>5/28/1990</Birthdate>
    <SSN>520-52-4275</SSN>
    <AccountNumber>5696618825</AccountNumber>
    <RoutingNumber>575159859</RoutingNumber>
    <Address>8348 Court Ave.</Address>
    <City>Pittsburgh,</City>
    <State>PA.</State>
    <Zip>15201</Zip>
  </Payroll>

 CREATE TABLE ReservationData (
  `id` mediumint(8) unsigned NOT NULL auto_increment,
  `UniqueID` MEDIUMINT default NULL,
  `TripDate` varchar(50) default NULL,
  `FirstName` varchar(255) default NULL,
  `LastName` varchar(255) default NULL,
  `Phone` varchar(100) default NULL,
  `AltPhone` varchar(100) default NULL,
  `Email` varchar(255) default NULL,
  `StreetAddress` varchar(255) default NULL,
  `City` varchar(50) default NULL,
  `State` varchar(50) default NULL,
  `Zip` varchar(10) default NULL,
  `Country` varchar(255) default NULL,
  `DayOfYear` varchar(50) default NULL,
  `TotalCost` varchar(50) default NULL,
  `Balance` varchar(10) default NULL,
  `CCard` varchar(18) default NULL,
  `Expires` varchar(5) default NULL,
  `CVC2` varchar(3) default NULL,
  PRIMARY KEY (`id`)
) TYPE=MyISAM AUTO_INCREMENT=1;

This would make great honey for a honey pot; just fill an SQL database with this random, realistic looking data, serve and log any and all attempts to access, query or dump the database. This can be done on a VM and you have a easily deployed, high interaction honeypot!

Aside from being able to see their IP address, I think the most useful data that can be attained is their behavior; what injection attacks are they using to drop the database? Write rules to prevent your honey against trivial attempts such as the ' AND 1=(SELECT attacks and see what they come up with next. Rule writing is inherently a cat-and-mouse game, honeypots like this clearly give the white-hats the upper hand.



Implementation



A quick, fast and dirty solution is to simply read a random line from a text file (i.e. Name_First.txt and Address_Street.txt).

This way, you can choose from names that are common, or customize your list to for different nationalities.

One could read the whole file in to a string, Parse() it into an array of strings, then randomly select an index, but this would be unacceptable for very large files. Instead, we can set the file pointer to a random position that is less than its size, roll back to the last new line and call ReadLine.




public string ReturnRandomLine(string FileName)
{
 string sReturn = string.Empty;
 
 using(FileStream myFile = new FileStream(FileName,FileMode.Open,FileAccess.Read))
 {
  using(StreamReader myStream = new StreamReader(myFile))
  {
   // Seek file stream pointer to a rand position...
   myStream.BaseStream.Seek(rand.Next(1,myFile.Length),SeekOrigin.Begin);
   
   // Read the rest of that line.
   myStream.ReadLine();
   
   // Return the next, full line...
   sReturn = myStream.ReadLine();
  }
 }
 
 // If our random file position was too close to the end of the file, it will return an empty string
 // I avoided a while loop in the case that the file is empty or contains only one line
 if(System.String.IsNullOrWhiteSpace(sReturn)) {
  sReturn = ReturnRandomLine(FileName);
 }
 
 return sReturn;
}

Example use:


public string GenerateFistName()
{
 return ReturnRandomLine("Name_First.txt") + " ";
}

public string GenerateLastName()
{
 return ReturnRandomLine("Name_Last.txt");
}

public string GenerateFullName()
{
 return GenerateFistName() + GenerateLastName();
}

public string GenerateGender()
{
 if(ReturnPercent(84)) {
  return "Male";
 } else {
  return "Female";
 }
}

public string GenerateStreetNumber()
{
 return rand.Next(1,9999).ToString();
}

public string GenerateStreetName()
{
 return ReturnRandomLine("Address_Street.txt");
}

One limitation is where the data is relational, such as in the case of generating a random zip code along with the city and state that it exists in.

A quick work-around would be CityZipState.txt



Other types of data that can be generated that would not make sense to put in a text file:


public bool ReturnPercent(int Percent) // Return true Percent times out of 100, randomly
{
 int iTemp = rand.Next(1,101);
 
 if(iTemp<=Percent) {
  return true;
 } else {
  return false;
 }
}

public string GenerateDate(int YearFrom,int YearTo)
{
 int Month = rand.Next(1,13);
 int Day  = rand.Next(1,32);
 int Year = GenerateYear(YearFrom,YearTo);
 
 return Month.ToString() + "/" + Day.ToString() + "/" + Year.ToString();
}

public string GenerateYear(int YearFrom,int YearTo)
{
 return rand.Next(YearFrom,YearTo+1).ToString();
}

public string GeneratePhoneNumber()
{
 return GeneratePhoneNumber(ReturnRandomLine("PhoneNumber_Prefix.txt"));
}

public string GeneratePhoneNumber(string Prefix)
{
 int iThree = rand.Next(192,999);
 int iFour = rand.Next(1000,9999);
 
 return Prefix + iThree.ToString() + "-" + iFour.ToString();
}

public string GenerateSSN()
{
 int iThree = rand.Next(132,921);
 int iTwo = rand.Next(12,83);
 int iFour = rand.Next(1423,9211);
 return iThree.ToString() + "-" + iTwo.ToString() + "-" + iFour.ToString();
}

Obviously, these methods can be improved to conform to the standards of a real social security number, national identification number, credit card number, ect...


public string GenerateCCNum()
{
 string sCCNum = string.Empty;
 
 byte[] bCCNum = {0};
 rand.NextBytes(bCCNum);
 
 // generate random 16 digit number
 int iTemp1 = rand.Next(10000000,99999999);
 int iTemp2 = rand.Next(10000000,99999999);
 string sTemp = iTemp1.ToString() + iTemp2.ToString();
 
 // while loop?
 while(!IsValidNumber(sTemp))
 {
  iTemp1 = rand.Next(10000000,99999999);
  iTemp2 = rand.Next(10000000,99999999);
  sTemp = iTemp1.ToString() + iTemp2.ToString();
 }
 
 sCCNum = sTemp;
 
 return sCCNum;
}

The implementation of IsValidNumber() is left as an exercise for the reader.

The serialization of your data is a trivial matter. Please see my post on a XML Serializable Dictionary, Tuple, and Object for the code to serialize an object (such as a list, or a class).