Limitations on exclusive copyrights for educational purposes generally fall under Fair Use. One must look at the Four Factor test to determine if it can be used in an in person educational setting. See Fair Use for details about the Four Factors.
§ 107 · Limitations on exclusive rights: Fair use
Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include—
(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
(2) the nature of the copyrighted work;
(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or value of the copyrighted work.
The fact that a work is unpublished shall not itself bar a finding of fair use if such finding is made upon consideration of all the above factors.
(Title 17 of the United States Code, Chapter 1, Section 107)
If a use does not fall under fair use, consider looking for works in the public domain, those under an open license (Creative Commons, open access, etc.), or seek permission from the copyright holder.
Additionally, certain performances and displays are allowed for educational use:
§ 110 · Limitations on exclusive rights: Exemption of certain performances and displays
Notwithstanding the provisions of section 106, the following are not infringements of copyright:
(1) performance or display of a work by instructors or pupils in the course of face-to-face teaching activities of a nonprofit educational institution, in a classroom or similar place devoted to instruction, unless, in the case of a motion picture or other audiovisual work, the performance, or the display of individual images, is given by means of a copy that was not lawfully made under this title, and that the person responsible for the performance knew or had reason to believe was not lawfully made;
Online classes may use fair use or the TEACH Act when determining if a use is legal. See the Four Factor Test or the TEACH Act Checklist.
If a use does not fall under fair use or is not in compliance with the TEACH Act, consider looking for works in the public domain, those under an open license (Creative Commons, open access, etc.), or seek permission from the copyright holder.