The GPL is not capable of "infecting" software it talks to over a network, so no, you do not have to disclose your server-side code.
In fact, this is closely related to why the Affero GPL exists:
The GNU Affero General Public License is a modified version of the
ordinary GNU GPL version 3. It has one added requirement: if you run a
modified program on a server and let other users communicate with it
there, your server must also allow them to download the source code
corresponding to the modified version running there.
The AGPL does not "infect across networks" either; even this extra condition is only meant to close the "but the user never runs it" loophole for server-side AGPL'd code. So your client-side Javascript library being GPL or LGPL or AGPL or whatever does not create a disclosure requirement for any server-side code. Only if your server-side code was linked to an AGPL library would you have to make it available.
Edit: I just noticed this section in the FSF's GPL FAQ is highly relevant:
Q: A company is running a modified version of a GPL'ed program on a web
site. Does the GPL say they must release their modified sources?
(#UnreleasedMods)
A: The GPL permits anyone to make a modified version
and use it without ever distributing it to others. What this company
is doing is a special case of that. Therefore, the company does not
have to release the modified sources.
It is essential for people to have the freedom to make modifications
and use them privately, without ever publishing those modifications.
However, putting the program on a server machine for the public to
talk to is hardly “private” use, so it would be legitimate to require
release of the source code in that special case. Developers who wish
to address this might want to use the GNU Affero GPL for programs
designed for network server use.