WASHINGTON—The Supreme Court on Wednesday will consider a case that could severely limit the Securities and Exchange Commission’s use of its special in-house courts to punish wrongdoers.

Critics have complained about the SEC’s special courts for years, saying they rob defendants, including those accused of fraud, of their right to a jury trial. Administrative law judges, who preside in the in-house courts, are supposed to be independent but are appointed by the same commissioners who authorize the enforcement cases that the judges hear.