Showing posts with label C#. Show all posts
Showing posts with label C#. Show all posts

C# Parse Negative Number When Using Double.Parse(String, NumberStyles)


The Problem
Our C# application sends a query to Solr server, and parses the response and generate graph report. Today the application throws error: Input string was not in a correct format. in one test environment.

At first thought, we thought it is due to the language and region settings, as shown in the post: C# Parsing is Regional(Culture) Sensitive

But we found out that in customer environment, it doesn't always fail, just failed in some rare cases. 
The Analysis
We re-executed the Solr stats query, and found some unexpected numbers in the response: the min value of the stats query is negative. This should not happen in normal cases. 

The real problem here is why these negative value comes from, and we should reject invalid value when push data to Solr.  We will fix the previous problem. 

But why it fails when C# parses the response? As the same code is used when parse all double value from solr response. It may contain negative value.

Checked the code and run it with the negative number.

string value = "-10.01";
double dvalue = double.Parse(value, System.Globalization.NumberStyles.AllowExponent | System.Globalization.NumberStyles.AllowDecimalPoint);
Console.WriteLine(dvalue);
It failed. Now it's clear that this is because Double.Parse(String, NumberStyles).
From Double.Parse Method (String, NumberStyles)
Converts the string representation of a number in a specified style to its double-precision floating-point number equivalent.

As we only specify AllowExponent and AllowDecimalPoint. It will disallow sign symbol. It will only take non negative value. Give it negative value, it will fail with exception. This code should be updated to use Double.parse(String s).

The Double.Parse Method allows string in format: [ws][sign][integral-digits[,]]integral-digits[.[fractional-digits]][E[sign]exponential-digits][ws]

Resources
C# Parsing is Regional(Culture) Sensitive
Solr: Extend StatsComponent to Support stats.query, stats.facet and facet.topn

C# Parsing is Regional(Culture) Sensitive


The Problem
Our C# application sent a query to Solr server, and parsing the response. It works fine, until recently one of our customers hits one exception:
Exception Message [Input string was not in a correct format.]
System.Number.ParseDouble(String value, NumberStyles options, NumberFormatInfo numfmt)
at System.Double.Parse(String s, NumberStyles style)

The Root Cause
Check the language settings, the server machine is using Finnish:fi-FI. 
The Solr response always uses en-US locale format for numbers, it uses "." period as decimal separator:
<double name="min">10.01</double>

But in fi-FI[Finnish(Finland)], it uses "," comma as decimal separator.

This would cause double.parse fail to parse the string like: 10.01, and throws exception: 
Input string was not in a correct format.

The fix is easy: using en-US culture when do double.parse: double.Parse("10.01", new CultureInfo("en-US")); or set en-US culture as the current culture of current thread:
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");

The workaround before we release the fix is to change the region and language settings to en_US for the user who starts the application.

Test Code
using System.Globalization;
using System.Threading;
// Add it via "add reference"
using Microsoft.VisualStudio.TestTools.UnitTesting;

static void Main(string[] args)
{
    CultureInfo originalCulture = Thread.CurrentThread.CurrentCulture;
    Console.WriteLine("Orignal Culture: " + originalCulture);
    CultureInfo enUS = CultureInfo.CreateSpecificCulture("en-US");

    CultureInfo fiFi = CultureInfo.CreateSpecificCulture("fi-FI");
    testParse(enUS, fiFi);
    testToString(enUS, fiFi);
    
    Console.ReadLine();
}
private static void testToString(CultureInfo enUS, CultureInfo fiFi)
{
    Thread.CurrentThread.CurrentCulture = fiFi;
    double d = 0.2;
    // output fi-FI format: 0,2
    String str = d.ToString();
    Console.WriteLine("fi-FI: " + d.ToString());
    Assert.AreEqual("0,2", str);

    str = d.ToString(enUS);
    // output en-US format: 0.2
    Console.WriteLine("fi-FI: " + str);
    Assert.AreEqual("0.2", str);
}
private static void testParse(CultureInfo enUS, CultureInfo fiFi)
{
    Thread.CurrentThread.CurrentCulture = fiFi;
    // print 10,01
    Console.WriteLine(10.01);
    double d1 = double.Parse("10,01");

    // use en-US to parse, this works fine.
    double d2 = double.Parse("10.01", enUS);
    try
    {
        // the follwing code would cause exception:  Input string was not in a correct format.
        double d3 = double.Parse("10.01");
        Assert.Fail("should fail");
    }
    catch (Exception e)
    {
        Console.WriteLine("expected ex: " + e);
    }
    // change culture to en-US
    Thread.CurrentThread.CurrentCulture = enUS;
    double d4 = double.Parse("10.01");
}

Update:
Today, we meet similar error, but this time is when C# sends a double string to java application: the region is actually de-AT. C# sent 0,2 to java servlet which is unable to parse it.

The solution is to convert to en-US: d.ToString(enUS)

Lesson Learned
When trouble shoot this problem in customser case, I noticed when I type some character, it is converted to other word, at that time, I should notice the customer is using a different language setting.

How to check Windows Region and Language Setting
Checking the Regional and Language Settings

Resources:
Java Locale “Finnish (Finland)” (fi-FI)
Using Language Identifiers (RFC 3066) - Region Code List

Labels

ANT (6) Algorithm (69) Algorithm Series (35) Android (7) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) JSON (7) Java (186) JavaScript (27) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) adsense (5) bat (8) regex (5) xml (5)