Musings of a Software Developer

Wednesday, April 22, 2009

Detect Prime numbers in one line – Python Code

The following code can be used to print all the prime numbers in a given range. With my recent obsession with Python, I learnt to admire the compactness of the code and all it takes to list the primes is just one line of code!

        1: inp = int(raw_input('Enter the outer bound : '))

       2: l = range(1,inp)

       3: #factors = [i for i in range(2,item/2+1) if item % i == 0]

       4: primes = [item for item in l if item>1 and len([i for i in range(2,item/2+1) if item % i == 0])==0]

       5: print primes

The code could further be optimized to improve the performance drastically.

       1: primes=[item for item in l if item>1 and len([i for i in range(2,8) if item%i==0])==0]
  

Notice that I am using list comprehensions and in the inner list comprehension inside the len() I am limiting the range of divisors to 8. Obviously any number greater than 8 would be divisible by one of the numbers between 2 and 8 (if it is a composite number).

The above optimized version is fast – so fast that to compute the list of primes between 1 and 50000, it just takes at most a second (did not time it actually) whereas the first one takes forever (i completed typing the whole paragraph and it is still running!). So I verified it with a range from 1 to 5000 and the first one is almost instantaneous.

But! there is a bug in the second algorithm that i listed. It does not verify those numbers that are divisible by other prime numbers which are not between 2 and 8. So any ideas to optimize this?

Basic algorithm: return all items in a list whose length of the factors list is 0.

Where is the place for optimization?

Computing the factors! is the key here. The faster you compute your factors, the better your algorithm would perform.

Sunday, April 05, 2009

Exploring Embedded Databases for using in ASP.NET

Recently, from the past couple of days I have been obsessed with finding an embedded database that supports concurrent read/write operations and which is easy to deploy. The following are the ones that I have come across and as we see, I share my personal opinion on what issues I think I might face if i ever have to use it in a ASP.NET application which might potentially have multiple users performing read/write on the database. Let me start with the ones I would not be using

SQL Server compact edition

I have seen mixed reviews about this database. I found Ayende personally moved to SSCE from SQLite as his choice of embedded database and he is having some issues. If he is unable to resolve this, I dont think I will ever be able to.

Moreover, the licensing of SSCE is intriguing though it says FREE. I personally did not get a nice feel about it.

The benefits : use entity framework, LINQ2SQL straight off the box from Visual Studio 2008.

Personal Verdict? Don’t care.

Firebird SQL

Some say, its good and some does not think so. Anyway the issue I have with this one is that one needs to go through a lot of workarounds to make it work with ASP.NET application and they say “try not to use this one in ASP.NET application”. Seriously – I believe 8 out of 10 applications people try to make are web applications and is intended to have multiple read/writes so they expect the database to have workaround the Reader/Writer scenario. Moreover, I decided I will stick to using NHibernate + FluentNHibernate as my DAL which makes me not feel good about using Firebird. I love FluentNHibernate and always disliked configurations using XML. I think, if something changes in your system then why not recompile your system and test it. Is it so hard?

Benefits? FirebirdSQL supposedly provides what SQLite cannot – better concurrency and “no file based locking”.

Issues? Oh! Lot many. First of all, there is a DDEX which they claim to work (did not work for me after spending two hours and yeah this has been the only thing I could not get to work in the past few months - but the documentation sucks big time). No one has ever bothered to write about using Firebird with C# applications (from what I searched) and those who have written did not do a great job, given the lack of proper official documentation. Another issue is that the work somehow is progressing very slow and this made me think that future releases are way too far to see any of my issues to be fixed. Moreover, FluentNHibernate does not support Firebird straight off the box. I need to write hibernate.cfg.xml and pass it on to the Configure() method which I don’t think I would want to do.

Personal Verdict? Don’t care.

ScimoreDB

Sounds scary to be used in an application that I plan to support for a long time. The documentation was decent but I don’t think it is that popular with the NHibernate folks out there. And being a lazy person that I am, I do not want to spend time mapping entities with tables.

Benefits? Not entirely clear about what to write – but the few folks who used it says its fast. Havent bothered trying it.

Issues? No ORM tutorial!! :)

VistaDB – commercial, so forget about it. Anyway, they say it is very good if you are willing to pay for it. If it was free, then I might have spent time to write my own mini ORM for this one (if there wasnt NHibernate support)

EasyDB – they had a dnrTV session but the site disappeared –> Another reason why i think everyone in the blogging community is smarter than me. Stick to SQLite!!

Now the bigger part of the blog – using SQLite in ASP.NET scenario.

Working with SQLite in ASP.NET environment.

Now, for those who are here looking for an answer to SQLite issue with file-level locking, you might be disappointed. Anyway, I did play with SQLite with NHibernate (ofcourse used FluentNH). It looks good for my sample tests and haven’t seen any major issues yet – because I haven’t written anything complex yet.

Issues? Folks says there is no support for concurrent read/write situations since the locks are implemented on file-level instead of table-level. So a writer is blocked out if there are any readers and readers are blocked out if there is a writer. But i hope to find some answer. I saw this page on the SQLite documentation which I plan to test out pretty soon. As of now, I have no idea on how to make it work from NHibernate using System.Data.SQLite.

Benefits? This is by far the most widely accepted embedded database that I have seen during my research and I personally like the support for .NET in the form of excellent .NET providers that Robert has come up with.

What else? I came across others like SharpHSQL (project looks dead), one from IBM and few other paid ones – I don’t want to pay at this moment.

What next?

I have had some questions whose answers I wish to find out either looking or trying. Once I have them, I would post them very soon. Also I would like to post a simple Database Interface wrapper using Fluent NHibernate which I used in my sample applications. The following is the road map for what I will be doing next. There might a couple of posts about them here.

1. One of the discussions on SQLite forums suggested that we serialize all read/write operations to a single thread. I need to figure out doing this on ASP.NET applications where having a background thread is not a good idea since the application pool is recycled killing all my threads.

2. If the above option does not work, then I have to play with the Shared Cache mode.

3. Tweaking and squeezing the best out of NHibernate for concurrent usages.

4. What is the best way to have NHibernate sessions in an ASP.NET application. Right now, I create a session as soon as a new ASP.NET session starts and flush it when the session ends. Theoretically it should be a good choice, but I need to dig more into it.

If you have found out anything more interesting than what I wrote here, please feel free to leave me a comment.

Anyway my final choice of free/embeddable database for my ASP.NET application would be SQLite. I would somehow figure out a way to allow concurrent read/write operations and will post here if i figure out something.

Thursday, March 26, 2009

Creating icons in InkScape

Because of my recent obsession to update my skills on Design (I think I am a decent programmer and with the design skills on my resume, it would be nice!), I created my first icon using Inkscape.

Ok – this isnt anyway close to the normal most stupid looking icons but hey I have an excuse – I am not a designer.

Anyway, as you work on Inkscape and try exporting your work as icon, I could not notice any export format as .ico. So here is what i did. I exported my work as a PNG (File->Export as Bitmap..) and then using http://www.convertico.com/ I converted my PNG to an icon!

For tutorial on working with inkscape

http://screencasters.heathenx.org/episode-055/

change episode-055 from episode-001 onwards.

Monday, March 23, 2009

Visual Studio Team Explorer: Going online!

Sometimes, it so happens that the source controlled project no longer shows its bindings to the source control and if you wish to get back the bindings, it is simple.

In the solution explorer, right click on the solution and click “Go Online”. then you can resume your normal work!

Tuesday, March 17, 2009

Some helper methods … extension methods …

In this post, I would like to document a couple of classes with several methods that are useful to convert from one type of data to another. For example, some of the methods can be used to convert a Dataset into JSON string and vice versa.

SQL Query output as XML

The first class to start with the most simplest one. This class can be used to wrap your SQL queries for Sql Server or Oracle such that they return XML output instead of rows. Strategy pattern has been used to implement these methods – nothing fancy.

The base class for query wrapper.

        1: public abstract class QueryWrapper

       2: {

       3:     public abstract string ForXmlOutput(string query);

       4: }

Then one wrapper each for Sql and Oracle has been implemented as shown.

       1: public class SqlQueryWrapper : QueryWrapper

       2: {

       3:     public override string ForXmlOutput(string query)

       4:     {

       5:         return (string.Format("{0} FOR XML RAW;", query)).Trim();

       6:     }

       7: }

       8:  

       9: public class OracleQueryWrapper : QueryWrapper

      10: {

      11:     public override string ForXmlOutput(string query)

      12:     {

      13:         return string.Format("SELECT DBMS_XMLGEN.getXML(\"{0}\",0) from dual;", query);

      14:     }

      15: }

And then there is services class that uses strategy pattern.

       1: public class WrapperServices

       2: {

       3:     private QueryWrapper _wrapper;

       4:     public WrapperServices(QueryWrapper wrapper)

       5:     {

       6:         this._wrapper = wrapper;

       7:     }

       8:  

       9:     public string WrapQueryForXml(string query)

      10:     {

      11:         return _wrapper.ForXmlOutput(query);

      12:     }

      13: }

Conversion classes

The following class provides methods for conversion between

1. XML to JSON

2. JSON to XML

3. Object to JSON

4. JSON to Object

5. Dataset to JSON

6. JSON to Dataset

7. XDocument to XmlDocument

8. XmlDocument to XDocument

9. Some dummy conversions to size In MB, time in seconds which might not be of much interest to you all.

For JSON methods, you would require a reference to JSON.NET libraries.

       1: public static class ConversionHelpers

       2:     {

       3:         #region Xml-X Document Conversions

       4:         /// <summary>

       5:         /// Converts XmlNode into an XDocument.

       6:         /// </summary>

       7:         /// <param name="doc">The XMLDocument to be converted</param>

       8:         /// <returns>The XDocument generated</returns>

       9:         public static XDocument AsXDocument(this XmlNode doc)

        {

            return XDocument.Load(new XmlNodeReader(doc));

        }

  

        /// <summary>

        /// converts the XNode into XmlDocument

        /// </summary>

        /// <param name="xdoc">The xdoc.</param>

        /// <returns>the xml document after conversion</returns>

        public static XmlDocument AsXmlDocument(this XNode xdoc)

        {

            var doc = new XmlDocument();

            doc.Load(xdoc.CreateReader());

            return doc;

        }

        #endregion

  

        #region Xml To JSON

        /// <summary>

        /// Converts the XNode passed into JSON String

        /// </summary>

        /// <param name="doc">Xdocument to be converted</param>

        /// <returns>JSON String</returns>

        public static string AsJsonString(this XNode doc)

        {

            return JavaScriptConvert.SerializeXmlNode(doc.AsXmlDocument());

        }

  

        /// <summary>

        /// converts the XML passed into json string.

        /// </summary>

        /// <param name="xml">The XML.</param>

        /// <returns>JSON String notation</returns>

        public static string AsJsonString(this string xml)

        {

            return XDocument.Parse(xml).AsJsonString();

        }

  

        /// <summary>

        /// Opens the XML file mentioned in "fi" and return it as JSON String.

        /// </summary>

        /// <param name="fi"></param>

        /// <returns></returns>

        public static string GetJsonContent(this FileInfo fi)

        {

            if (File.Exists(fi.FullName))

                return XDocument.Load(fi.FullName).AsJsonString();

            else

            {

                return null;

            }

        }

  

        #endregion

  

        #region Json to XML

        /// <summary>

        /// Converts the json string into XML

        /// </summary>

        /// <param name="json">JSON as string</param>

        /// <returns>JSON returned as XML</returns>

        public static string AsXml(this string json)

        {

            return JavaScriptConvert.DeserializeXmlNode(json).InnerXml;

        }

        #endregion

  

        #region Object <-> JSON

        public static string ToJSON<T>(this T obj)

        {

            return JavaScriptConvert.SerializeObject(obj);

        }

  

        public static T FromJSON<T>(this string json)

        {

            return JavaScriptConvert.DeserializeObject<T>(json);

        }

        #endregion

  

        #region Dataset <--> JSON

        public static string ToJSON(this DataSet ds)

        {

            return ds.GetXml().AsJsonString();

        }

  

        public static DataSet ToDataset(this string jsonString, bool isXml)

        {

            var ds = new DataSet();

            string asXml = string.Empty;

            if (!isXml)

                asXml = jsonString.AsXml();

            else

            {

                asXml = jsonString;

            }

            using (var sr = new StringReader(asXml))

            {

                ds.ReadXml(sr);

            }

            return ds;

        }

  

        #endregion

  

        #region Others....

        /// <summary>

        /// Computes the size of the byte array in Megabytes

        /// </summary>

        /// <param name="array">byte array to be computed</param>

        /// <returns>size of the array in MB</returns>

        public static double SizeInMB(this byte[] array)

        {

            return array.Length / (1024 * 1024);

        }

  

        /// <summary>

        /// Converst the value into seconds. Basically it divides by 1000.

        /// </summary>

        /// <param name="val">Value in Milliseconds</param>

        /// <returns>Value in seconds</returns>

        public static double TimeInSeconds(this double val)

        {

            return val / (1000);

        }

  

        #endregion

    }

Test Data Helpers

In this set of helper classes, I wrote a couple of methods which wrap the most required functionality (at least for me) like generating a random string, getting a random integer, timing an execution. May or may not be useful to you all.

       1: /// <summary>

       2: /// Provides methods for timing execution of an action.

       3: /// Also provides method to generate a byte[] array with all random values.

       4: /// Other Random Helper methods.

       5: /// </summary>

       6: public class TestHelpers

       7: {

       8:     private static Random rand = new Random();

       9:     /// <summary>

    /// This method executes the action passed and then returns the total time

    /// taken in milliseconds using Stopwatch.

    /// </summary>

    /// <param name="action">the action to be performed</param>

    /// <returns>time elapsed in milliseconds</returns>

    public static double TimeAndExecute(Action action, int repeatCount)

    {

        if (repeatCount == 0) return 0;

  

        Stopwatch sw = Stopwatch.StartNew();

        for (int i = 0; i < repeatCount; i++)

            action();

        sw.Stop();

  

        return sw.ElapsedMilliseconds / repeatCount;

    }

  

    /// <summary>

    /// Returns a random byte array of the specified size.

    /// </summary>

    /// <param name="size"></param>

    /// <returns></returns>

    public static byte[] GetBytes(int size)

    {

        var b = new byte[size];

        var rand = new Random(size);

        rand.NextBytes(b);

        return b;

    }

  

    public static int NextIntRand()

    {

        return rand.Next();

    }

  

    /// <summary>

    /// returns a randomly generated string.

    /// </summary>

    /// <param name="length">length of the string to be returned</param>

    /// <returns>string generated</returns>

    public static string GetRandomString(int length)

    {

        return Encoding.Default.GetString(GetBytes(length));

    }

}

Simplified Xml querying using Extension Methods.

The following method returns all the elements in the given document (XDocument) with the passed ElementName and which have the specified attribute name and value. I use it a lot for automating my XML documents manipulation.

       1: public static class XDocumentExtensionMethods

       2: {

       3:     public static IEnumerable<XElement> GetDescendantsWithCondition(this XDocument lgxDoc, string ElementName, string AttributeName, string AttributeValue)

       4:     {

       5:         return lgxDoc.Descendants(ElementName).Where(it =>

       6:         {

       7:             XAttribute attribute = it.Attribute(AttributeName);

       8:             return (attribute != null &&

       9:                     attribute.Value == AttributeValue);

      10:         });

      11:     }

      12:  

      13: }

Below are few more helper methods which I use regularly. One of them is the GetCopy() method which returns a copy of the passed XElement.

       1: class XDocumentHelpers{

       2: /// <summary>

       3: /// Modifies the element attribute.

       4: /// </summary>

       5: /// <param name="elementToModify">The element to modify.</param>

       6: /// <param name="attributeName">Name of the attribute.</param>

       7: /// <param name="attributeValue">The attribute value.</param>

       8: public static void ModifyElementAttribute(XElement elementToModify, string attributeName, string attributeValue)

       9: {

    if (elementToModify == null || String.IsNullOrEmpty(attributeName)) Debug.WriteLine("Passed element is NULL");

    else

    {

        XAttribute attribute = elementToModify.Attribute(attributeName);

        if (attribute == null) elementToModify.SetAttributeValue(attributeName,attributeValue);

        else

        {

            attribute.SetValue(attributeValue);

        }

    }

}

  

/// <summary>

/// Adds the element attribute.

/// </summary>

/// <param name="elementToModify">The element to modify.</param>

/// <param name="attribute">The attribute.</param>

/// <param name="attributeValue">The attribute value.</param>

public static void AddElementAttribute(XElement elementToModify, string attribute, string attributeValue)

{

    if (elementToModify == null || String.IsNullOrEmpty(attribute)) Debug.WriteLine("Passed element is NULL");

    else

    {

        if (elementToModify.Attribute(attribute) == null)

        {

            XAttribute xa = new XAttribute(attribute, attributeValue);

            elementToModify.Add(xa); //add the attribute

        }

        else

            ModifyElementAttribute(elementToModify, attribute, attributeValue);

    }

}

  

/// <summary>

/// Deletes the element attribute.

/// </summary>

/// <param name="elementToModify">The element to modify.</param>

/// <param name="attrToDelete">The attr to delete.</param>

public static void DeleteElementAttribute(XElement elementToModify, string attrToDelete)

{

    if (elementToModify == null || String.IsNullOrEmpty(attrToDelete)) Debug.WriteLine("Passed element is NULL");

    else

    {

        XAttribute attribute = elementToModify.Attribute(attrToDelete);

        if (attribute == null) Debug.WriteLine("Could not find the attribute " + attrToDelete);

        else attribute.Remove();

    }

}

  

public static void DeleteNode(XElement element)

{

    if (element == null) return;

    element.Remove();

}

  

public static XElement GetCopy(XElement mainTable)

{

    return XElement.Parse(mainTable.ToString());

}

}

I hope these utility methods provides good information about JSON, XML, XLinq, etc. Let me know if you have any comments to share.