I am starting to work with class libraries in my C# class. We were told to write code that can be used more than one time. Should I keep all my class definitions in one class library for the length of the term, or separate them by project?
Separate them by purpose. A class library is supposed to be cohesive. You shouldn't throw arbitrary things together.
For example, I just wrote an updater library. It has classes that deal with updating an application. But it doesn't have classes for some neat string operations because it doesn't make sense to add them. Any time I want to reuse some code, I want to reuse just one thing (for example this updater). If you throw everything together, it will be a mess and you will also force yourself to reference the whole thing when you really want to use just one. Give yourself the freedom to reuse the parts you want.
Writing truly reusable code is hard, but I definitely encourage you to try and do this. You will learn important things about writing actually reusable code.
Most enterprise code-bases I've seen organise their classes into different projects based on their purpose. For example, within one solution, you might have the following projects:
SampleSolution.Presentation (User-interface things)
SampleSolution.BusinessLogic (Business rules/logic things)
SampleSolution.Persistance (Data-related things)
There are many variations on the particular name given to the library... For example, I've seen the BusinessLogic project named Domain, BusinessRules, Business... Just depends on your preference really. But the main thing is separating these items out makes things more maintainable and logical - it's a lot easier to quickly find the class you're looking for. Depending on the way the app is being developed, there might also be additional libraries such as Models, Views, Controllers, DTOs (Data transfer objects), and so on.
Depending on the nature of your assignment(s), it may or may not make sense to separate them up like this. For example, if you're going to be writing some classes which do things like manipulate strings, or perform calculations - those that fall under a "utility" kind of categorisation - it would make a lot of sense to put these all in the one class library. This way, in future assignments, you can just reference the compiled DLL, which is how you would typically use third-party libraries.
I use actual Class Libraries (i.e. DLLs) by purpose/architecture, and then further separate those using folders and namespaces. For example in a typical app I might have:
Wayne.MyApplication.Domain (Business objects)
Wayne.MyApplication.Core (Common things required by everything else)
and then further split them up (e.g. my Domain project contains folders for Models, Repositories, and Services). One thing to be wary of is the "DDD Anti-pattern" whereby you have a separate project for your Domain objects, DTOs, Repositories, Services, Specifications, etc. etc. etc. when those are logical separations, not physical ones.