We investigate electroabsorption (EA) in organic semiconductor microcavities to understand whether strong light-matter coupling non-trivially alters their nonlinear optical [\$\$\\chi \ˆ\(3)\łeft(ømega,\\\\\mathrm\0,0\\\\\\right)\$\$] response. Focusing on strongly-absorbing squaraine (SQ) molecules dispersed in a wide-gap host matrix, we find that classical transfer matrix modeling accurately captures the EA response of low concentration SQ microcavities with a vacuum Rabi splitting of \$\$\hslash Ømega \approx 200\$\$meV, but fails for high concentration cavities with \$\$\hslash Ømega \approx 420\$\$meV. Rather than new physics in the ultrastrong coupling regime, however, we attribute the discrepancy at high SQ concentration to a nearly dark H-aggregate state below the SQ exciton transition, which goes undetected in the optical constant dispersion on which the transfer matrix model is based, but nonetheless interacts with and enhances the EA response of the lower polariton mode. These results indicate that strong coupling can be used to manipulate EA (and presumably other optical nonlinearities) from organic microcavities by controlling the energy of polariton modes relative to other states in the system, but it does not alter the intrinsic optical nonlinearity of the organic semiconductor inside the cavity.