2011 April

LMAO @ Sony…

So I finally got ‘the email’ from PSN, about freakin’ time, 7 days late if you ask me.

===================================

PlayStation(R)Network

===================================

Valued PlayStation(R)Network/Qriocity Customer:

We have discovered that between April 17 and April 19, 2011,
certain PlayStation Network and Qriocity service user account
information was compromised in connection with an illegal and
unauthorized intrusion into our network. In response to this
intrusion, we have:

1) Temporarily turned off PlayStation Network and Qriocity services;

2) Engaged an outside, recognized security firm to conduct a full
and complete investigation into what happened; and

3) Quickly taken steps to enhance security and strengthen our
network infrastructure by rebuilding our system to provide you
with greater protection of your personal information.

We greatly appreciate your patience, understanding and goodwill
as we do whatever it takes to resolve these issues as quickly and
efficiently as practicable.

Although we are still investigating the details of this incident,
we believe that an unauthorized person has obtained the following
information that you provided: name, address (city, state, zip), country,
email address, birthdate, PlayStation Network/Qriocity password and login,
and handle/PSN online ID. It is also possible that your profile data,
including purchase history and billing address (city, state, zip),
and your PlayStation Network/Qriocity password security answers may
have been obtained. If you have authorized a sub-account for your
dependent, the same data with respect to your dependent may have
been obtained. While there is no evidence at this time that credit
card data was taken, we cannot rule out the possibility. If you have
provided your credit card data through PlayStation Network or Qriocity,
out of an abundance of caution we are advising you that your credit
card number (excluding security code) and expiration date may have
been obtained.

For your security, we encourage you to be especially aware of email,
telephone and postal mail scams that ask for personal or sensitive
information. Sony will not contact you in any way, including by email,
asking for your credit card number, social security number or other
personally identifiable information. If you are asked for this information,
you can be confident Sony is not the entity asking. When the PlayStation
Network and Qriocity services are fully restored, we strongly recommend that
you log on and change your password. Additionally, if you use your PlayStation
Network or Qriocity user name or password for other unrelated services or
accounts, we strongly recommend that you change them as well.

To protect against possible identity theft or other financial loss, we
encourage you to remain vigilant, to review your account statements and
to monitor your credit reports. We are providing the following information
for those who wish to consider it:
- U.S. residents are entitled under U.S. law to one free credit report annually
from each of the three major credit bureaus. To order your free credit report,
visit www.annualcreditreport.com or call toll-free (877) 322-8228.

- We have also provided names and contact information for the three major U.S.
credit bureaus below. At no charge, U.S. residents can have these credit bureaus
place a “fraud alert” on your file that alerts creditors to take additional steps
to verify your identity prior to granting credit in your name. This service can
make it more difficult for someone to get credit in your name. Note, however,
that because it tells creditors to follow certain procedures to protect you,
it also may delay your ability to obtain credit while the agency verifies your
identity. As soon as one credit bureau confirms your fraud alert, the others
are notified to place fraud alerts on your file. Should you wish to place a
fraud alert, or should you have any questions regarding your credit report,
please contact any one of the agencies listed below:

Experian: 888-397-3742; www.experian.com; P.O. Box 9532, Allen, TX 75013
Equifax: 800-525-6285; www.equifax.com; P.O. Box 740241, Atlanta, GA 30374-0241
TransUnion: 800-680-7289; www.transunion.com; Fraud Victim Assistance Division,
P.O. Box 6790, Fullerton, CA 92834-6790

- You may wish to visit the website of the U.S. Federal Trade Commission at
www.consumer.gov/idtheft or reach the FTC at 1-877-382-4357 or 600 Pennsylvania
Avenue, NW, Washington, DC 20580 for further information about how to protect
yourself from identity theft. Your state Attorney General may also have advice
on preventing identity theft, and you should report instances of known or
suspected identity theft to law enforcement, your State Attorney General,
and the FTC. For North Carolina residents, the Attorney General can be
contacted at 9001 Mail Service Center, Raleigh, NC 27699-9001; telephone
(877) 566-7226; or www.ncdoj.gov. For Maryland residents, the Attorney
General can be contacted at 200 St. Paul Place, 16th Floor, Baltimore, MD 21202;
telephone: (888) 743-0023; or www.oag.state.md.us.

We thank you for your patience as we complete our investigation of this
incident, and we regret any inconvenience. Our teams are working around the
clock on this, and services will be restored as soon as possible. Sony takes
information protection very seriously and will continue to work to ensure that
additional measures are taken to protect personally identifiable information.
Providing quality and secure entertainment services to our customers is
our utmost priority. Please contact us at 1-800-345-7669 should you have any
additional questions.

Sincerely,

Sony Computer Entertainment and Sony Network Entertainment

Put simply: “YOU DON’T” so quit trying.

Just today:
http://stackoverflow.com/questions/5806520/parse-email-header-with-regex-in-c
http://thedailywtf.com/Articles/Email-Validation-Validity.aspx

Why people insist on attempting this is beyond me. I’ve grown to really like Regex, I’ve even built a light-weight xml parser on regular expressions but you can not parse email addresses with a regex. Just look at some of these crazy attempts in the daily wtf comments.

If you want to validate an email use:

 try
{ new System.Net.Mail.MailAddress(email); return true; }
catch(FormatException)
{ return false; }

If you MUST use a regular expression on the client for email validation, try this one:

.+@.+

More Info:
http://tools.ietf.org/html/rfc2822#section-3.4.1

3.4.1. Addr-spec specification

An addr-spec is a specific Internet identifier that contains a locally interpreted string followed by the at-sign character (“@”, ASCII value 64) followed by an Internet domain.

Yes, locally interpreted, meaning you can’t. Yes the RFC does state some constraints, but they are too complex to express in a regex. Get a library or don’t validate it.

Additions in this release:

Introduced CSharpTest.Net.BPlusTree.dll – a fairly full featured IDictionary<K,V> implementation backed by a B+Tree on disk.
Collections.LListNode<T> – a doubly linked list implementation that can support asynchronous iteration.
Collections.SynchronizedDictionary/SynchronizedList to support synchronization of a list/dictionary given a locking strategy from the Synchronization namespace.
IO.ClampedStream to provide an IO stream aggregation for a subset of the provided stream.
IO.Crc32 to provide calculation of a CRC32 value from bytes or strings.
IO.FileStreamFactory an IFactory<Stream> producer of streams for a given file.
IO.FragmentedFile an underpinning of the B+Tree implementation that provides sub-allocations within a single file.
IO.SharedMemoryStream a block allocating memory stream that can be simultaneously used by multiple threads at the same time.
IO.StreamCache a pool of open file streams that a thread can open and close without the overhead of actually opening or closing the underlying file streams.
Interfaces.IFactory<T> provides a simple generic factory interface for supplying instances of type T.
Interfaces.ITransactable provides a simple transaction interface.
IpcChannel.IpcEventChannel provides a cross domain/process connectionless communication built on events. see this SO post.
Serialization.ISerializer<T> provides a simple interface for an object that can read and write an instance of type T to and from a stream.
Serialization.PrimitiveSerializer provides basic implementation of the ISerializer<T> interface for the primitive types.
Serialization.VariantNumberSerializer provides a protobuffer-like encoding for numeric types.
Threading.WaitAndContinueList a work list based on WaitHandles and resulting actions so that multiple activities can be performed on a single thread.
Threading.WaitAndContinueWorker a single worker thread that processes a WaitAndContinueList.
WorkQueue and WorkQueue<T> provide simple thread pool processing of tasks that the caller can wait for completion on.
Utils.ObjectKeepAlive a simple object to track references to other instances to avoid garbage collection.
Utils.WeakReference<T> a derivation of WeakReference that is type-safe.
Synchronization classes are newly rewritten.

Breaking changes in this release:

The Synchronization namespace has undergone a complete overhaul. If your currently depending upon it’s interfaces or implementation you may want to stay with the version you have until you can determine the impact. Some simple uses of the previous classes may still work, but this a complete rewrite. Why? Simply put the last version was junk. The added cost of the abstraction layer was more than the lock itself. I’ve retooled it to avoid new instances on lock, removed the use of TimeSpan, removed the upgrade locks, and simplified the interfaces. The end result is a very clean interface that is easy to use and fast.

Cont’d from Building a database in C# – Part 2

Caching and persistence seem to go hand-in-hand for this adventure. With the cost of serialization and IO it’s just not possible to get anything to perform well without a little caching. The questions I had around caching is really more along the lines of “how” and “what” not about “if”. Let’s be clear, I generally hate using cache. It often produces ugly and problematic code that is hard to follow and even harder to debug. There are cases where it makes sense to use a cache and I felt this was one of those.

First things first, answering “What do I cache?”

This is a fairly obvious answer given the B+Tree structure. I need to cache these ‘node’ things and their associated lock. The biggest question that came up was if I wanted to exclude the data-nodes from the cache and only cache the hierarchy nodes. Conceptually this would improve the cache hits as the tree structure will be used many times more than the leaf data nodes. However, as I experimented with this I came to two issues with the exclusion: 1) The cost to rehydrate data nodes can be more expensive than the hierarchy nodes and to exclude it may reduce overall performance. 2) The cache mechanic I have in place is also responsible for caching the lock instance for a given node.

Next answering the hard question “How do I cache?”

I decided to tackle this as a three-prong approach. I wanted to provide several cache mechanics, one that does not cache data, one that caches everything, and then strike a balance somewhere in the middle. For obvious reasons all of these are write-through caching. The first was fairly easy, I needed only a lookup to provide the lock instance used for the node and to access the node itself I continually pass-through to the storage interface. The ‘full/complete’ cache was also trivial and performs very well if you happen to be able to fit all your data in memory. The usefulness of this seems limited since though since the whole point here is that the data is too large to fit entirely in memory. After all if I could fit it all in memory I might prefer to just use a Dictionary and simply save the contents to disk. So this complete cache is cool and all but doesn’t serve us well, so where is the balance?

Caches in a database tend to follow these basic rules: 1) allocate a hoard of memory, 2) sub-allocate from it until it is depleted, 3) once depleted try and unload something. This is a reasonable approach for most C++ applications, especially dedicated database processes. However, I’m building an embedded database/storage engine not a dedicated database server. The idea of allocating 100mb or 1gb at startup seems absurd in the managed world. After all I have something those C++ guys don’t have, a Garbage Collector. The trick to leveraging it to house the cache is that I need a few things to happen: 1) I can’t hold a direct reference to everything so the GC can clean up, and 2) I need to hold a direct reference to enough stuff that the GC won’t constantly clear my entire cache. So the first is easy enough, I’m sure your all aware of the WeakReference class and it’s properties so I won’t go into detail there. For the second part of this, keeping references alive, I needed some code…

Enter the ObjectKeepAlive class. I needed a high-throughput class that will hold references for me, but what criteria to use to keep reference around? Well to keep things easy I went with a simple min/max range for the number of items and timespan that decides when to reduce the number from max to min. Thus I always keep at least min items alive and have at most max items that are no older than the timeout. This was fairly easy to put together as a forward-only linked list of arrays where the list stores a time-stamp of last modification. I actually started by modifying the last example lock-less queue I posted here. I’ll have the code updated soon and for those wanting to join me abusing the GC you might find the ObjectKeepAlive class useful ;)

I know these posts have not been very detailed and stay on vague side. Unfortunately it is difficult to deep dive some of these topics without brutally long backgrounds on both concepts and implementation details. With the implementation spanning 1.3k LOC across 20 files it’s more involved than anyone in their right mind would care to read about. As it is I hope the code will prove more useful than these topics, I promise it’s coming soon, it’s already at the self-imposed required 100% functional coverage. I’m only waiting on sorting out some issues with the migration to mercurial over on googlecode.com.

Well I went looking around for an XSD schema that could validate XML comments I place in code for documentation. Either Microsoft never delivered one, or I completely failed to find it. Amazing that I found many people looking for it, but nobody willing to write it. So I did, you can get it here:

http://csharptest.net/downloads/schema/XmlDocumentation.xsd

The schema will validate all of xml comments in the .NET BCL 2.0 through 4.0 so it should be close enough for whatever you have in mind. Some of the Microsoft specific stuff (PermissionSet) I kinda punted on as I couldn’t find anything consistent about what they were producing. Other than that though it should be complete enough.

Enjoy!

No idea how I found myself on this blog; however, it was an interesting read: Why Is 100% Test Coverage Easier To Achieve? Although the translation is a little difficult to read he does an excellent job of pointing out the obvious benefits to 100% functional coverage. Let’s recap his main points… 1. First he [...]

csharptest.net

Important information regarding PlayStation Network and Qriocity services

How to correctly validate an email address with a regex (Regular Expressions)

Version v1.11.426.305 released

Building a database in C# – Part 3

XSD for Xml Comments for .NET Documentation

RE: Why Is 100% Test Coverage Easier To Achieve?

Search

Projects

Recent Updates