Glembatumumab Vedotin in Treating Patients With Metastatic or Locally Recurrent Uveal Melanoma

Inclusion Criteria:

* Patients must have histologically or cytologically confirmed metastatic or locally recurrent uveal melanoma; because histologic or cytologic confirmation of primary uveal melanoma is not always possible, confirmation of the clinical diagnosis of uveal melanoma by the treating investigator is allowed; clinical diagnosis of uveal melanoma is often made by an ophthalmologist, not by tissue diagnosis; if an ophthalmologist diagnosed and treated a patient for uveal melanoma in the past, it is sufficient for a clinical diagnosis
* Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) as \>= 20 mm (\>= 2 cm) with conventional techniques or as \>= 10 mm (\>= 1 cm) with spiral computed tomography (CT) scan, magnetic resonance imaging (MRI), or calipers by clinical exam
* The study will be limited to patients who are chemotherapy naive; patients may have received prior systemic or liver-directed local therapies for advanced uveal melanoma as long as those treatments do not involve chemotherapy; this includes, but is not limited to: immunotherapy, targeted therapy, transarterial embolization, radiofrequency ablation, or cryoablation; treatment must be completed at least 28 days prior to initiation of study therapy; radiation therapy is also allowed and must be completed at least 28 days prior to initiation of study therapy; lesions treated via radiation or liver-directed therapy may not be used as target lesions unless they demonstrate growth over a minimum of 3 months on subsequent imaging
* All prior treatment-related toxicities must be Common Terminology Criteria for Adverse Events (CTCAE) version (V) 5 grade =\< 1 (except alopecia); certain exceptions apply, such as immunotherapy-induced hypothyroidism or adrenal insufficiency or panhypopituitarism requiring stable doses of hormone replacement or rash from prior therapy
* Eastern Cooperative Oncology Group (ECOG) performance status =\< 2
* Life expectancy of greater than 3 months
* Leukocytes \>= 3,000/uL
* Absolute neutrophil count \>= 1,500/uL
* Platelets \>= 100,000/uL
* Total bilirubin =\< 1.5 x institutional upper limit of normal
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal; =\< 5 x institutional upper limit of normal if liver metastasis present
* Creatinine =\< institutional upper limit of normal OR creatinine clearance \>= 60 mL/min/1.73 m\^2 for patients with creatinine levels above institutional normal
* Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and for 4 months after last CDX-011 (glembatumumab vedotin) dose; women of child-bearing potential must have a negative serum pregnancy test within 14 days prior to start of protocol treatment; should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately; men and women treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 4 months after completion of glembatumumab vedotin administration
* Ability to understand and the willingness to sign a written informed consent document

Exclusion Criteria:

* Patients with a history of another malignancy except for those who have been disease-free for 2 years; patients with a history of definitively treated non-melanoma skin cancer or squamous cell carcinoma of the cervix are eligible; patients with definitively treated in-situ cancers are eligible, regardless of timeframe
* Patients with neuropathy \> grade 1
* Patients who are receiving any other investigational agents; if the patient received a previous investigational or other agent or treatment, a washout period of 4 weeks is required
* Patients receiving any medications or substances that are substrates of cytochrome P450 family 3, subfamily A, polypeptide 4 (CYP3A4) will be closely monitored for toxicity; as part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Human immunodeficiency virus (HIV)-positive patients on antiretroviral medications that are CYP3A4 substrates will be closely monitored; HIV-positive patients will be excluded if they have a cluster of differentiation 4 (CD4) count \< 200
* Pregnant or nursing women
* Patients who have previously received CDX-011 (glembatumumab vedotin) or other monomethyl auristatin E (MMAE)-containing agents
* Patients with a history of allergic reactions attributed to compounds of similar composition to dolastatin or auristatin (e.g. Auristatin PHE, Auristatin PE, and symplostatin)