A realy good way I've seen for organizing exceptions in a Ruby library is in sferik's twitter library.

https://github.com/sferik/twitter/blob/master/lib/twitter/error.rb

By using Ruby's dynamic class creation mechanism with `Class.new(ParentClass)`
It makes it very easy to reason about the class hierarchy.

Client related exceptions inherit from ClientError.
Server related exceptions inherit from ServerError.
ClientError and ServerError both inherit from Twitter::Error

HTTP response codes are mapped to error classes and raised upon receiving the HTTP response:

    @parser = Http::Parser.new(http_response)
    error = Twitter::Error::ERRORS[@parser.status_code]
    raise error if error

I've abreviated the code to only show the important parts:

    module Twitter
      class Error < StandardError
        ...    
        ClientError = Class.new(self)
        BadRequest = Class.new(ClientError)
        Unauthorized = Class.new(ClientError)
        RequestEntityTooLarge = Class.new(ClientError)
        NotFound = Class.new(ClientError)
        NotAcceptable = Class.new(ClientError)
        UnprocessableEntity = Class.new(ClientError)
        TooManyRequests = Class.new(ClientError)
        
        Forbidden = Class.new(ClientError)
        AlreadyFavorited = Class.new(Forbidden)
        AlreadyRetweeted = Class.new(Forbidden)
        DuplicateStatus = Class.new(Forbidden)

        
        ServerError = Class.new(self)
        InternalServerError = Class.new(ServerError)
        BadGateway = Class.new(ServerError)
        ServiceUnavailable = Class.new(ServerError)
        GatewayTimeout = Class.new(ServerError)

        ERRORS = {
          400 => Twitter::Error::BadRequest,
          401 => Twitter::Error::Unauthorized,
          403 => Twitter::Error::Forbidden,
          404 => Twitter::Error::NotFound,
          406 => Twitter::Error::NotAcceptable,
          413 => Twitter::Error::RequestEntityTooLarge,
          422 => Twitter::Error::UnprocessableEntity,
          429 => Twitter::Error::TooManyRequests,
          500 => Twitter::Error::InternalServerError,
          502 => Twitter::Error::BadGateway,
          503 => Twitter::Error::ServiceUnavailable,
          504 => Twitter::Error::GatewayTimeout,
        }.freeze
        ...
      end
    end