32

I know that many of us maintain our own little personal library with tools and utilities that we use often.

I've had mine since I was 16 years old so it has grown to quite a considerable size. Some of the stuff I've written has since been added to the framework. I wrote my own little implementation of expression trees for use with genetic algorithms long before LINQ, which I quite like and was proud of at the time - of course its pretty useless now. But recently I have been going through it and upgrading to .NET 4.0 and re-kindled an interest.

So I'm curious as to what you use your library for. Maybe we could get some cool ideas going for useful little snippets and share them amongst ourselves.

So my questions are:

  • Do you have a miscellaneous utility library?
  • Which part are you most proud of and why?

Give an example of code if you like :-)

7
  • Nobody seems to be upvoting answers...
    – Joey Adams
    Dec 7, 2010 at 20:13
  • @Joey Adams Right? Currently 17 question votes and 6 total answer votes.
    – Nicole
    Dec 7, 2010 at 21:07
  • I'm not really seeing answers worth upvoting. What does an upvote mean for them? The nature of the question is such that the answers just get a "oh. nice." kind of reaction, and then it's either upvote everything or nothing at all. (And I don't like upvoting every answer just cause it's there. If nothing else, I'm out of votes. :P)
    – Adam Lear
    Dec 7, 2010 at 21:40
  • @Anna Lear, ok, you're excused :)
    – Nicole
    Dec 7, 2010 at 22:16
  • 3
    Any decent utility should be put up on github and shared with the world. It does not make sense to keep it hidden if it is truly good.
    – Job
    Dec 8, 2010 at 1:36

16 Answers 16

28

No.

I've seen some nightmarish effects of a dozen developers all adding their own little "util.h" style libraries to projects, and have it turn into a giant mess of inconsistent function naming and behaviors. Much like PHP. So for that reason I avoid doing it.

I avoid needing to do that by using programming environments that give me nearly all the tools and libraries I need up front whenever possible, such as C# and python.

8
  • 7
    I constantly rewrite my library for organizational purposes.
    – Maxpm
    Dec 7, 2010 at 19:37
  • 4
    Cases where the utils package has turned nightmarish does not mean that all are bad. I can't see how you can avoid it and not have more code duplication because of it. And therefore, worse testing and less efficiency.
    – Nicole
    Dec 7, 2010 at 21:09
  • 3
    @Renesis: The utils packages are about as disastrous as goto statements. Sure, by itself its not all that bad, but it just seemingly always ends up being a disaster sooner or later. As for code duplication, if you find yourself doing some similar task in virtually all your projects, then for something like python or C#, other people have probably been doing it too and it's probably in the standard libraries by then. Dec 7, 2010 at 21:21
  • 7
    In my experience, engineers with their own library will favour using it ahead of the system-provided one, so it is not good practice to have personal libraries. I once had a guy who was absolutely convinced that his 'strlen' function was faster than the compiler-provided one, because he wrote it. It took a simple demonstration of how strlen is a couple of inlined assembly instructions for him to concede that maybe other people can do better. Dec 8, 2010 at 0:03
  • 4
    @JBRWilkinson Your point is well taken. Not every programmer is fit to develop common code.
    – Nicole
    Dec 8, 2010 at 1:37
16

SmartFormat

My favorite utility is one I wrote - a simple string builder/formatter that makes it really easy to turn data into strings with correct grammar.

For example, most programmers build text from a template: "There are {0} items remaining" but this leads to grammatical errors: "There are 1 items remaining".

So, SmartFormat lets you write: "There {0:is|are} {0} item{0:|s} remaining".

You just replace String.Format(...) with Smart.Format(...) and that's it!

The SmartFormat code is open source: http://github.com/scottrippey/SmartFormat/wiki

6
  • This reminds me the format used by java.text.MessageFormat.
    – barjak
    Dec 9, 2010 at 12:54
  • @barjak Interesting! I spent a long time researching "conditional" formatting, and never found anything similar until now! MessageFormat has the ChoiceFormat class that allows a SURPRISINGLY similar syntax! An example from the documentation: "There {0,choice,0#are no files|1#is one file|1<are {0,number,integer} files}.". Thanks for mentioning this reference. Feb 15, 2011 at 21:19
  • @barjak Just to validate my point, though, SmartFormat has many more features! Conditional formatting works for any data type, such as bool, date, timespan, and object; it also supports advanced operators, such as "{Count:<0?negative|=5?five|>50&<100?large|other}". It has reflection (ie "There are {items.Length} items", can format array items and timespans. Plus, it has a plugin model for supporting even more features. Feb 15, 2011 at 21:30
  • It seems to be powerful, indeed. The arrays formatting is interresting.
    – barjak
    Feb 16, 2011 at 11:06
  • @barjak: Yeah, array formatting is really useful! Check out this example: Smart.Format("There are {0.Count} files: {0:'{}'|, |, and }.", files); would result in "There are 3 files: 'a.txt', 'b.txt', and 'c.txt'.". I can't imagine "localization" without it. Feb 16, 2011 at 20:57
7

K Combinator (C#, Scala)

I use the K combinator in Ruby quite often, mostly in folds when the folding operation is performed through a side effect rather than a return value, like in this example:

some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1 }

This counts how often each element occurs in some_collection. Unfortunately, it doesn't actually work, since the block has to return the new value of the accumulator at each iteration, but in Ruby assignments evaluate to the assigned value.

So, you have to ecplicitly return the new value of the accumulator like this:

some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1; acc }

But I find such explicit sequencing ugly in this functional-ish style using folds. The K combinator (called Object#tap in Ruby) to the rescue:

some_collection.reduce(Hash.new(0)) {|acc, el| acc.tap { acc[el] += 1 }}

I have already missed it a couple of times in C# (mostly because for some reason collection mutators such as List.Add return void instead of this) and Scala, so I carry around this:

namespace GenericExtensions
{
    public static class GenericExtensions
    {
        public static T Tap<T>(this T o, Action<T> f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f(o);
            return o;
        }

        public static T Tap<T>(this T o, Action f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f();
            return o;
        }
    }
}

and in Scala:

class Tap[T](o: T) {
  def tap(f: T => Unit) = { f(o); o }
  def tap(f: => Unit) = { f; o }
}

object Implicits { implicit def any2Tap[T](o: T) = new Tap(o) }

Identity Function (Ruby)

Something I am missing in Ruby, is a nicely named way to access the identity function. Haskell provides the identity function under the name of id, Scala under the name of identity. This allows one to write code like:

someCollection.groupBy(identity)

The equivalent in Ruby is

some_collection.group_by {|x| x }

Doesn't exactly roll off the tongue, does it?

The fix is

IDENTITY = -> x { x }

some_collection.group_by(&IDENTITY)

ForEach (.NET)

Another sorely missing method in C#:

namespace IEnumerableExtensions
{
    public static class IEnumerableExtensions
    {
        public static void ForEach<T>(this IEnumerable<T> xs, Action<T> f)
        {
            Contract.Requires(xs != null);
            Contract.Requires(f != null);

           foreach (var x in xs) f(x);
        }
    }
}
10
  • 3
    I think your last example was a calculated design decision. The concept of an Action implies side-effects which goes against the design principles of LINQ. Dec 7, 2010 at 18:53
  • 1
    @ChaosPandion: What does this have to do with LINQ? Dec 7, 2010 at 18:58
  • @Jörg W Mittag - The IEnumerable extensions were added for LINQ. Dec 7, 2010 at 19:37
  • 2
    @ChaosPandion: I still don't understand. ForEach is not a LINQ operator. Why should restrictions that only apply to LINQ operators apply to ForEach, which is not a LINQ operator? And why are side-effects forbidden for IEnumerable.ForEach but allowed for List.ForEach? Also, why are side-effects forbidden for IEnumerable.ForEach but allowed for foreach? Dec 7, 2010 at 20:58
  • @Jörg W Mittag - What I am saying is the fact that it is missing from the extensions was a design decision. The fact that List<T> has a ForEach is reasonable considering that it is a mutable type. Dec 7, 2010 at 21:02
6

I have a Java Type Converter. It has a public signature

public static <T> T convert(Object sourceValue, Class<T> destinationType)

and it does its best to convert the source value to the destination type. It essentially lets you do dynamic typing within a statically typed language :-)

It's actually useful with boxed numeric types. How irritating it is that you can't put an Integer to where Long is expected? No problem, just convert it. Or what if your function expects a double, but you have a null to put there? Kaboom, a NPE. But put it through convert, and you get a NaN.

2
  • Interesting solution. I've always kind of thought that Long should extends Integer. But even then you'd still have the autoboxing problem (as far as I know there's no way autoboxing would work with the inheritance). Also, +1 for NaN support.
    – Nicole
    Dec 8, 2010 at 7:34
  • NaN is excellent. Too bad that there's no such thing for integers. I've used Integer.MIN_VALUE as a convention. It's usually "weird enough" to be noticed, unlike the default value 0. I don't know why auto(un)boxing doesn't treat (Double) null as NaN. It's the obvious right solution, IMHO. Dec 8, 2010 at 8:08
6

Of the misc code I've written, most of the good stuff is in CCAN now, while the rest I tend to find better versions of in existing open source projects. I find myself writing less and less general-purpose "misc" code these days, in favor of writing application-specific variants of such code, or of writing general-purpose modules that I can release by themselves.

C

Here's a function and typedef I've used more than once. For applications that need timing, it's hard to beat milliseconds in terms of simplicity:

#include <stdint.h>
#include <sys/time.h>

typedef int64_t msec_t;

static msec_t time_ms(void)
{
    struct timeval tv;
    gettimeofday(&tv, NULL);
    return (msec_t)tv.tv_sec * 1000 + tv.tv_usec / 1000;
}

And more miscellaneous C functions that I tend to use over and over (and over):

/* Remove a trailing newline, if present. */
void chomp(char *buffer)
{
    if (!*buffer)
        return;

    while (*buffer)
        buffer++;

    if (buffer[-1] == '\n')
        buffer[-1] = 0;
}

/*
 * Skip whitespace, update the pointer, and return it.
 * Example:
 *
 * switch (*skipSpace(&s)) {
 *     case '\0':
 *         ...
 *     case '(':
 *         ...
 */
const char *skipSpace(const char **sptr)
{
    const char *s = *sptr;
    while (isspace(*s))
        s++;
    *sptr = s;
    return s;
}

/* Scramble an array of items uniformly. */
void scramble(void *base, size_t nmemb, size_t size)
{
    char *i = base;
    char *o;
    size_t sd;
    for (;nmemb>1;nmemb--) {
        o = i + size*(rand()%nmemb);
        for (sd=size;sd--;) {
            char tmp = *o;
            *o++ = *i;
            *i++ = tmp;
        }
    }
}

Haskell

Haskell's nub :: (Eq a) => [a] -> [a] function is O(n²) because, by its type signature, it's only allowed to test if two elements are equal. A simple O(n log n) alternative is map head . group . sort, but it requires forcing the entire input list before producing output, whereas nub can start producing output right away. The following is an O(n log n) alternative to nub that collects already-seen items in a Data.Set:

module Nub (nub') where

import Prelude
import Data.Set (empty, member, insert)

nub' :: Ord a => [a] -> [a]
nub' xs = loop xs empty where
    loop [] _ = []
    loop (x:xs) set =
        if x `member` set
            then loop xs set
            else x : loop xs (insert x set)

In Haskell, I use alternatives to sequence, mapM, forM, replicateM, and filterM. These actions each generate a list, but the list can't be used until the action completes in its entirety (if you're using a strict monad like IO). The alternatives build the list in reverse rather than forming a tower of thunks, which I found through benchmarking to be faster, at least with GHC.

sequence' :: Monad m => [m a] -> m [a]
sequence' ms = loop ms [] >>= return . reverse where
    loop []     xs = return xs
    loop (m:ms) xs = do
        x <- m
        loop ms (x:xs)

mapM' :: Monad m => (a -> m b) -> [a] -> m [b]
mapM' f xs = sequence' $ map f xs

forM' :: Monad m => [a] -> (a -> m b) -> m [b]
forM' = flip mapM'

replicateM' :: Monad m => Int -> m a -> m [a]
replicateM' n x = sequence' (replicate n x)

filterM' :: Monad m => (a -> m Bool) -> [a] -> m [a]
filterM' pred xs = loop xs [] >>= return . reverse where
    loop []     xs' = return xs'
    loop (x:xs) xs' = do
        keep <- pred x
        loop xs (if keep then (x:xs') else xs')

Note: sequence_, mapM_, forM_, and replicateM_ functions are still a better choice if you aren't interested in the result list.

1
  • +1 for CCAN, though I could be considered a bit biased :)
    – Tim Post
    Dec 8, 2010 at 5:50
4

I wind up implementing split/join ala Perl in languages that don't have it.

I also have reimplemented atoi and itoa in C more times than I want to think about (embedded systems junk).

4

No.

I do most of my coding in Java, and best practice is to reuse "utils" from Apache Commons libraries and similar projects.

If you are objective about it, there are few cases where your own "utils" collection will be a significant improvement on what other people have already done. And if it is not an improvement, then your utils library is probably a waste of development time, and a nuisance / burden for future maintainers.

3

I had some date manipulations that I performed using Java, then I started using JodaTime as I had heard good things about it and its to be included in Java 7 (not sure if this is still the case, but even if its not it is still well worth using it imho).

It turned a 50+ line class into one line with about three chained method calls.

For the curious it involved getting the date for each day of n weeks past : eg the sales figure for a monday 10 weeks ago etc etc).

And here is part of it

public static DateTime getDayPreviousWeek(DateTime dt, DayOfWeek dayOfWeek, int n_weeks) {
       return dt.minusWeeks(n_weeks).dayOfWeek().setCopy(dayOfWeek.getDayAsString());
}
2
  • java has extension methods?
    – Kugel
    Dec 7, 2010 at 23:51
  • nope, but I think it might be getting them in version 7 Dec 8, 2010 at 9:40
2

I always have a utils package of some sort, even in Java, but my PHP utils collection is the most re-used. There are so many good libraries in Java, that I either already have a library included in the project or need to just design a few missing utils on my own. PHP libraries tend to do too much for me to want to include them in my projects.

I kind of like this function for PHP, refined with help on StackOverflow...

function getValueFromDotKey(&$context, $name) {
    $pieces = explode('.', $name);
    foreach ($pieces as $piece) {
        if (!is_array($context) || !array_key_exists($piece, $context)) {
            // error occurred
            return null;
        }
        $context = &$context[$piece];
    }
    return $context;
}

It is similar to Apache's BeanUtils for Java, and I use it for a similar purpose, giving form elements in a template language a single key that can get/set a nested value in a source array:

$source = array('a' => array('b' => 5));

$val = getValueFromDotKey($source, 'a.b');

Of course, being PHP, I wanted to keep the method as lightweight as possible so it isn't quite as featureful as BeanUtils ;)

2

Scala standard library lacks some most commonly used higher order functions.

Two such functions that I need most often:

// #1: unfold
def unfold[T, R](init: T)(f: T => Option[(R, T)]): List[R] = f(init) match {
  case None => Nil
  case Some(r, v) => r :: unfold(v)(f)
}

// #2: zipWith
def zipWith[A, B, C](xs: List[A], ys: List[B])(f: (A, B) => C): List[C] = {
  (xs, ys).zipped.map(f)
}
1

Currently, no. I had one when I was doing C, but now that I do Java, it no longer makes sense, considering all the standard libs available, plus all the goodies coming from the Apache project.

One of the useful things in my C lib was a quick&dirty finite state machine implementation, which allowed the definition of a finite state machine with just two strings and an array of strings. It could be used to check strings against rules (e.g. "must be 4..6 characters long, first one a letter, rest digits") , but the availability of regexes made that thingy completely pointless.

1

I can't write desktop UIs now without dynamic dialogs, based on differential execution. It's a hack I stumbled on around 1985, and I've re-implemented it in various languages more times than I can remember.

1

I found I was writing a lot of the same code in django, Do this common thing, then this common thing, and finally that common thing. Basically get one or more items from the database, or save the results of a form.

If Each one of these things occurs just once in a view, then I can use the django generic views. Unfortunately, those are not really composable, and I needed to do several things in sequence.

So I went and wrote an even more generic views library, one that worked by first building a list of actions from relevant querysets (or whatever), and then wrapped the list into a view.

I still have to write some views by hand, but these are usually complex enough that there's not much that's reusable in them. All of the boilerplate just lands elsewhere, either a generic view, or as a view decorator (often a decorated generic view). This typically ends up being about 10% of the handlers I write, since some generic handler can do everything else.

1

Yes, but only for domain-specific idioms structures (like game-objects-specific containers).

As it's simple utility tools than anything complex, I'm not proud of anything there. I'm the unique user at the moment anyway so there is nothing to be proud of.

1

C++ indirect sort, based on the STL sort and a functor template.

The need for indirect sorting (in which the desired output was the permutation indexes that would result from sorting the data, but not the sorted data itself) appeared many times in a number of projects. I always wondered why STL did not provide an implementation for it.

Another was a C++ cyclic vector, where positive and negative indexes are modulo with the vector size (so that any integer values are valid indexes for the vector).

-4

I wrote a small utils package when I was doing Java development in my Comp. Sci class in high school. I'm most proud of my random number generator.

/**
* Returns a random integer.
*
* @returns    int    Random integer
*/
public static int getRandom()
{
    return 4; // Chosen by a fair dice roll.
              // Guaranteed to be random.
}

Props to my inspiration.

9
  • 12
    c'mon, xkcd....
    – Darknight
    Dec 7, 2010 at 16:48
  • 2
    C'mon, it doesn't matter.
    – Josh K
    Dec 7, 2010 at 17:13
  • 8
    Plagiarism is the highest form of flattery, except when it's obvious.
    – Maxpm
    Dec 7, 2010 at 19:39
  • 5
    Well the downvote button says: "This answer is not usefull". I guess there is a need for an aditional button: "...but sure is funny"
    – skajfes
    Dec 7, 2010 at 21:44
  • 2
    +1 because I feel like it. And bc you added the link to xkcd
    – Nobody
    Dec 7, 2010 at 22:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.