Radiation Therapy With or Without Olaparib in Treating Patients With Inflammatory Breast Cancer

Inclusion Criteria:

* Patients must have inflammatory breast cancer without distant metastases. All biomarker subtype groups (estrogen receptor \[ER\], progesterone receptor \[PR\], HER2) are eligible. Inflammatory disease will be defined per American Joint Committee on Cancer (AJCC) 8th edition with documentation by history/exam and pathology at the time of diagnosis.
* All patients must have completed neoadjuvant chemotherapy prior to mastectomy. The chemotherapy regimen is at the discretion of the treating physician but it is recommended that it include at least 4 cycles of anthracycline and/or taxane-based therapy (plus targeted therapy for patients with HER2+ disease). Response to chemotherapy is not a criterion for eligibility (both complete responders and those with residual disease are eligible). Please note that although pathologic complete response (pCR) is not required or excluded, pCR status must be determined post-surgery prior to randomization.
* All patients must have undergone modified radical mastectomy (with negative margins on ink) with pathologic nodal evaluation (from level I and II axillary lymph node dissection \[ALND\]) at least 3 weeks and no more than 12 weeks prior to randomization, unless they receive additional chemotherapy after mastectomy. Patients must not have gross residual tumor or positive microscopic margins after mastectomy.
* Additional adjuvant chemotherapy after surgery is allowed at the discretion of the treating physician, either completed prior to randomization or planned for after completion of protocol treatment. If adjuvant chemotherapy is administered after mastectomy, the patient must be randomized at least 3 weeks but no more than 12 weeks after the last dose of adjuvant chemotherapy.
* Patients must not have a history of radiation therapy to the ipsilateral chest wall and/or regional nodes. Prior radiation therapy to other body sites is allowed.
* Patients must not be planning to receive any other investigational agents during radiation therapy. Prior therapy, including prior treatment with olaparib or other PARP inhibitor, is allowed.
* Patients must not have a known hypersensitivity to olaparib or any of the excipients of the product.
* Patients must not have unresolved or unstable grade 2 or greater toxicity (with the exception of alopecia) from prior administration of another investigational drug and/or prior anti-cancer treatment.
* Patients must not be planning to receive strong or moderate CYP3A inhibitors or inducers while on olaparib treatment. Patients receiving strong or moderate CYP3A inhibitors must agree to discontinue use at least 2 weeks prior to receiving olaparib. Patients receiving strong or moderate CYP3A inducers must agree to discontinue use at least 5 weeks prior to receiving olaparib.
* Patients must not be planning to receive live virus or live bacterial vaccines while receiving olaparib and during the 30 day follow up period
* Patients must not be planning to receive any additional anti-cancer therapy (chemotherapy, endocrine therapy, immunotherapy, biological therapy or other novel agent) while receiving radiotherapy with or without study medication. If a patient is receiving concurrent anti-HER2 targeted therapies, they must not take these medications during the period of radiotherapy (with or without study drug) while enrolled on the study.
* Patients must be \>= 18 years of age
* Patients must have Zubrod performance status 0-2.
* Absolute neutrophil count (ANC) \>= 1000/mm\^3 (within 28 days prior to registration)
* Platelet count \>= 100,000/mm\^3 (within 28 days prior to registration)
* Hemoglobin \>= 9.0 g/dL (after transfusion if required and within 28 days prior to registration)
* Patients must have adequate renal function as evidenced by calculated creatinine clearance \>= 51 mL/min by Cockcroft-Gault equation, within 28 days prior to registration.
* Total bilirubin =\< 1.5 x upper limit of normal (ULN) (within 28 days prior to registration)

  * Patients with documented Gilbert's disease may have bilirubin up to 2.5 mg/dL
* Serum glutamic-oxaloacetic transaminase (SGOT) =\< 2.5 x ULN (within 28 days prior to registration)
* Serum glutamate pyruvate transaminase (SGPT) =\< 2.5 x ULN (within 28 days prior to registration)
* Alkaline phosphatase =\< 2.5 x ULN (within 28 days prior to registration)
* Patients must not have a history of other prior malignancy except for the following: adequately treated basal cell or squamous cell skin cancer, in situ cervical cancer, adequately treated stage I or II cancer from which the patient is currently in complete remission, or any other cancer from which the patient has been disease free for five years
* Female patients must be postmenopausal or have a negative urine or serum pregnancy test within 14 days prior to registration. Female patients of childbearing potential (and male patients with female partners who are of childbearing potential or pregnant) who are sexually active, must agree to the use of two highly effective forms of contraception during protocol treatment and for 6 months following the last dose of olaparib. Note: The efficacy of hormonal contraceptives may be reduced if co-administered with olaparib. Male patients must agree not to donate sperm during protocol treatment and for 6 months after the last dose of olaparib.
* Patients who are breastfeeding must agree to discontinue breastfeeding before receiving olaparib due to potential risk for adverse events in nursing infants secondary to treatment of the mother with olaparib.
* Patients must not have active uncontrolled infection, symptomatic congestive heart failure, unstable angina pectoris or cardiac arrhythmia.
* Patients must be able to swallow and retain oral medications and have no known gastrointestinal disorders likely to interfere with absorption of the study medication.
* Patients must not have a history of a resting electrocardiography (ECG) indicating uncontrolled, potentially reversible cardiac conditions (such as unstable ischemia, uncontrolled symptomatic arrhythmia, congestive heart failure, Fridericia's formula corrected QT interval \[QTcF\] prolongation \> 500 ms, electrolyte disturbances) or congenital long QCYP3T syndrome.
* Patients must not have myelodysplastic syndrome (MDS)/acute myeloid leukemia (AML) or with features suggestive of MDS/AML.
* Patient must not have had major surgery within 2 weeks of starting study treatments and patients must have recovered from any effects of any major surgery.
* Patients must not have a history of uncontrolled ventricular arrhythmia, recent (within 3 months) myocardial infarction, uncontrolled major seizure disorder, unstable spinal cord compression, superior vena cava syndrome, or extensive interstitial bilateral lung disease on high resolution computed tomography (HRCT) scan.
* Patients must not have had previous allogenic bone marrow transplant or double umbilical cord blood transplantation (dUCBT).
* Patients must not have had whole blood transfusions in the last 120 days prior to randomization.
* Patients must be offered the opportunity to participate in specimen submission for banking.

  * Note: Germline and somatic BRCA status (genetic testing) are planned for future correlative evaluation, in order to examine treatment and circulating tumor deoxyribonucleic acid (ctDNA) response as stratified by BRCA 1/2 mutational status. Since this is future planned correlative research, any mutational status results would not be returned to the patient or the treating physician. There is no Clinical Laboratory Improvement Act (CLIA)-certified clinical genetic testing being performed for patients as part of the S1706 study. A forthcoming revision or separate corelative sciences proposal would be submitted to and approved by National Cancer Institute (NCI) prior to conduct of any planned future translational medicine objectives.
* Patients who can complete the patient-reported outcomes (PRO) Quality of Life (QOL) and PRO-CTCAE questionnaires in English must be offered the opportunity to participate in the optional PRO substudy. Patients who are not able to complete questionnaires in English need not be offered the opportunity to participate.
* Patients must be informed of the investigational nature of this study and must sign and give written informed consent in accordance with institutional and federal guidelines.
* As a part of the OPEN registration process, the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system.