Unidentified, undated - 45 of 48
Cite this item
-
Brief Collection, LDF Court Filings. Chance v. Board of Examiners Brief for Plaintiffs-Appellees, 1972. 73d62631-ad9a-ee11-be37-00224827e97b. LDF Archives, Thurgood Marshall Institute. https://ldfrecollection.org/archives/archives-search/archives-item/2f24b75b-8eb7-4219-bce5-3f0a25faa57e/chance-v-board-of-examiners-brief-for-plaintiffs-appellees. Accessed August 19, 2025.
Copied!
UNITED STATES COURT OF APPEALS
FOR THE SECOND CIRCUIT
NO. 71-2021
BOSTON M. CHANCE, LOUIS C.
MERCADO, et al..
Plaintiffs-Appellees,
against
THE BOARD OF EXAMINERS,
Defendant-Appellant,
and
THE BOARD OF EDUCATION OF THE CITY
OF NEW YORK, et al..
Defendants.
Appeal From An Order Of The United States District
Court For The Southern District Of New York
BRIEF FOR PLAINTIFFS-APPELLEES
JACK GREENBERG
JONATHAN SHAPIRO
ELIZABETH B. DUBOIS
10 Columbus Circle
New York, New York 10019
Phone: (212) 586-8397
GEORGE COOPER
435 West 116th Street
New York, New York 10027
MICHAEL O. FINKELSTEIN
26 Broadway
New York, New York
Attorneys for Plaintiffs-Appellees
TABLE OF CONTENTS
*1
Statement of The Issues Presented for Review
Statement of The Case ................
(1) The Proceedings Below ..............
(2) The District Court's Decision .....
(3) The Preliminary Injunction .........
(4) Subsequent Developments ............
ARGUMENT:
PAGES
1
2
2
12
17
18
Introduction .......................
Point I. The District Court's Finding That
Defendants' Examination System Had A
Significant And Substantial Discrimi
natory Impact Was Clearly Supported By The Record ..........................
Point II. The Court Below Correctly Ruled That A
Public Employer Violates The Equal
Protection Clause In Conducting A Pro
motional Examination System Which
Systematically And Significantly Dis
criminates Against Ethnic Minority Groups
Where That Examination System Cannot Be
Shown To Have Any Relation To Job Performance ........
A. A Prima Facie case Of Discrimination Is
Made Out Where Examinations Significantly
And Systematically Exclude Members Of
An Ethnic Minority Group Regardless Of
Whether Any Subjective Intent To Discriminate is Shown ..........
B. Where Employment Tests Are Shown To Have
A Significant And Substantial Discrimi
natory impact On Ethnic Minority Groups
They Violate The Equal Protection Clause
If They Cannot Be Shown To Be Job-Rela-
(1) The Court Applied The Proper
Standard In Determining That An
Adequate Case Of Discriminatory Impact Had Been Made Out .......
i
PAGE
POINT III.
(2) The Court Properly Determined That
Examinations Which Have Such A Discrimi
natory Impact Violate The Equal Pro
tection Clause If They Cannot Be Shown
To Be Job-Related ................ a.4
The Court's Finding That Defendants'
Examinations Had Not Been Shown To Be
4 And Apparently Were Not Job-Related Was
Clearly Supported By The Record ............ 52
v POINT IV. The Grant Of Preliminary Relief, Based
Upon The Court's Findings With Regard
To irreparable Injury And The Likeli
hood Of Success On The Merits, Must Be
Upheld As A Proper Exercise Of Dis
cretion ..................... 50
Conclusion ...
Appendix A ...
Appendix B ...
4
1 1
TABLE OF AUTHORITIES
Cases
Page
Armstead v. Starkville Mun. Sep. School Dist.,
325 F. Supp. 560 (N.D. Miss. 1971)...... 23,37,44,49,57
^Arrington v. Massachusetts Bay Transportation
Authority, 306 F. Supp. 1355 (D. Mass.
1969) ....................................... 36,45,49, 57
Baker v. Columbus Municipal Separate School
District, 329 F. Supp. 706 (N.D. Miss. 1971) .. 37,44,45,
49,57
Bolling v. Sharpe, 347 U.S. 497 (1954) ................ 47
Brown v. Allen, 344 U.S. 443 (1953) ................... 47
Carmichael v. Craven, No. 26,236, 9th Cir.,
Nov. 4, 1971 ....................................... 35
Carter v. Gallagher, 3 CCH E.P.D. 18205 (D. Minn.
1971), aff’d in pertinent part. 3 CCH E.P.D.
18335 (1971), aff'd in pertinent part en banc.
No. 71-1181 (8th Cir., Jan. 7, 1972) ...........36,39,41,
48,57,60
^Castro v. Beecher, 4 CCH E.P.D. 17569, clarified.
4 CCH E.P.D. 17570, -judgment modified. 4 CCH
E.P.D. 17589 (D. Mass. 1971) .................. 36,40,46
Chaney v. State Bar of California, 386 F.2d 962
(9th Cir. 1967) ................................... 44
Checker Motors Corp. v. Chrysler Corp., 405 F.2d 319
(2d Cir.), cert, denied. 394 U.S. 999 (1969) .... 59
Council of Supervisory Associations v. Board of
Education, 23 N.Y.2d 458, 297 N.Y.S.2d 547,
245 N.E. 2d 204 (1969) ................. ...... 22 62
Dandridge v. Williams, 397 U.S. 471 (1970) ............ 48
Dino de Laurentiis Cinematografica, S.P.A. v. D-150,
366 F . 2d 373 (2d Cir. 1966) .................. [___ 59
Gaston County v. United States, 395 U.S. 285 (1969)
iii
35
Goodwin v. Wyman, 330 F. Supp. 1038 (S.D. N.Y. 1971) .. 48
Page
Graham v. Richardson, 403 U.S. 365 (1971) ............. 47
Gregory v. Litton System, 316 F. Supp, 401 (Cent.
D. Cal. 1970) ...................................... 40
Briggs v. Duke Power Co., 401 U.S. 424 (1971),
reversing, 420 F.2d 1225 (4th Cir. 1970) 34,37,38,39,42,
47,48,49,57
Harper v. Virginia Board of Elections, 383 U.S.
663 (1966) ..................... ................... 47
Hawkins v. Town of Shaw, 437 F.2d 1286 (5th Cir. 1971) 35,48
Hicks v. Crown Zellerbach Corp., 319 F. Supp. 314
(E.D. La. 1970) ................................ 37,57,60
Hobson v. Hansen, 269 F. Supp. 401 (D. D.C. 1967),
aff'd sub nom. Smuck v. Hobson, 408 F.2d 175
(D.C. Cir. 1969) ............................... 35,42,48
Jackson v. Godwin, 400 F.2d 529 (5th Cir. 1968) ...... 35,48
Jackson v. Wheatley School District, 430 F.2d 1359
(8th Cir. 1970) .................................... 23
Johnson v. New York State Education Department,
449 F. 2d 871 (2d Cir. 1971) ...................... 48
Johnson v. Pike Corp., 4 CCH E.P.D. J7517 (Cent.
D. Cal. 1971) 37,40
Jones v. Georgia, 389 U.S. 24 (1967) .................. 41,47
Kennedy Park Homes Ass'n v. Lackawanna, 436 F.2d
108 (2d Cir. 1970) ................................ 35,47
Kirkpatrick v. Preisler, 394 U.S. 526 (1969) .......... 41
Korematsu v. United States, 323 U.S. 214 (1944) ...... 47
Kramer v. Union Free School District, 395 U.S. 621
(1969) 46
Loving v. Virginia, 388 U.S. 1 (1967) ................. 47
IV
Page
McLaughlin v. Florida, 369 U.S. 184 (1964) ............ 46,47
Norris v. Alabama, 294 U.S. 587 (1935) ................ 46
North Carolina Board of Education v. Swann,
402 U.S. 43 (1971) ................................ 35
Packard Instrument Company v. Ans. Inc., 416 F.2d
943 (2d Cir. 1969) ................................ 59
Patton v. Mississippi, 332 U.S. 463 (1947) ............ 47
Penn v. Stumpf, 308 F. Supp. 1238 (N.D. Cal. 1970) ___ 36,39
Pickens v. Okalona Mun. Sep. Sch. Dist., No.
EC6956-K (N.D. Miss., Aug. 11, 1971) (mimeo.
OP-) ............................................... 46,49
^orcelli v. Titus, 431 F.2d 1254 (3d Cir. 1970),
aff1 g, 302 F. Supp. 726 (D. N.J. 1969) ........... 22,23
Powell v. Power, 436 F.2d 84 (2d Cir. 1970) ........... 35
Robinson v. Lorillard Corp., 444 F.2d 791 (4th Cir.
1971) ........................................ 37,45,46, 50
Schware v. Board of Bar Examiners, 353 U.S. 232 (1957) 44
Shapiro v. Thompson, 394 U.S. 618 (1969) ............... 47
Sims v. Georgia, 389 U.S. 404 (1967) ................... 41
Smith v. Texas, 311 U.S. 128 (1940) .................... 35
Societe Comptoir v. Alexander's Dept. Stores, 299 F.2d
33 (2d Cir. 1962) .................................. 59
Southern Alameda Spanish Speaking Org. v. Union City,
424 F . 2d 291 (9t.h Cir. 1970) ....................... 35
Turner v. Fouche, 396 U.S. 346 (1970) ................. 41
United States v. Bethlehem Steel, 446 F.2d 652
(2d Cir. 1971) ..................................... 46,50
United States v. Jacksonville Terminal Co., 3 CCH
E.P.D. f8324 (5th Cir. 1971) .................. 45, 50,57
v
United States v. W. T. Grant Co., 345 U.S. 629 (1953) . 59
United States ex rel. Chestnut v. Criminal Court of
City of New York, 442 F.2d 611 (2d Cir. 1971) ___ 41
Wells v. Rockefeller, 394 U.S. 542 (1969) ............. 41
W e s t e r n Addition Community Organization v. Alioto,
330 F. Supp. 536 (N.D. Cal. 1971) ........... .. 36,40,46
Whitus v. Georgia, 385 U.S. 545 (1967) ................. 41
̂ Statutes and Regulations
California Fair Employment Practices. CCH E.P. Guide
120,861 ............................................ 43
Colorado Civil Rights Commission Policy Statement on
the Use of Psychological Tests. CCH E.P. Guide
121,060 ............................................ 43
EE0C Guidelines on Employee Selection Procedures.
29 CFR §1607, 35 Fed. Reg. 12333 (Aug. 1, 1970) 43,57,60
N.Y. State CONST. Art. V §6 ............................ 44
N.Y. State Educ. Law §2569(1) (1967) .................. 44
' OFCC Guidelines on Employee Selection Procedures.
35 Fed. Reg. 19307 (Oct. 2, 1971) ................ 43
V
Pennsylvania Guidelines on Employee Selection Proce-
dures, CCH E.P. Guide 15194 ....................... 43
42 U.S.C. §1981 ......................................... 2
42 U.S.C. §1983 ..................................... 2
42 U.S.C. §2000-e .................... „
Page
vi
Ot h e r A u t h o r 1 t i e s
ANASTASI, PRINCIPLES OF PSYCHOLOGICAL TESTING (3d
Ed. 1968) ......................................... 56, 57
Center for Field Research and School Services, New
York University, A Report of Recommendations in
the Recruitment. Selection, Appointment and
Promotion of Teachers in the New York City
Public Schools (1966) ............................ 24
COLEMAN, EQUALITY AND EDUCATIONAL OPPORTUNITY (1966) . 42
Cooper and Sobol, Seniority and Testing under Fair
Employment Laws. 82 HARV. L. REV. 1598 (1969) ... 42
CRONBACH, ESSENTIALS OF PSYCHOLOGICAL TESTING
(3d Ed. 1970)...................................... 54
Commission on Human Rights for the City of N. Y.,
Egual Employment Opportunity and the New York
City Public Schools, An Analysis and Recommenda
tions Based on Public Hearings Held January 25-29.
M ........................... .................. 25
CRESAP, McCORMICK and PAGET, Summary Report of
Assignments Conducted for the New York City
Board of Education (1962) ........................ 24
FISS, A Theory of Fair Employment Laws. 38 U. CHI.
L. REV. 235 (1971) ................................ 24
GHISELLI, THE VALIDITY OF OCCUPATIONAL APTITUDE
TESTS (1966) ...................................... 42
GRIFFITH REPORT, Teacher Mobility in New York City:
A Study of Recruitment. Selection, Appointment.
and Promotion of Teachers in the New York City
Public Schools (1963) ............................ 24
KIRKPATRICK, et al., TESTING AND FAIR EMPLOYMENT (1968) 42
Mayor's Advisory Panel on Decentralization of the
New York City Schools, Reconnection for
Learning (1967) [BUNDY REPORT]..................... 24
Mayor's Committee on Management Survey, Administra
tive Management of the School System of New York
City (1951) ["STRAYER AND YAVNER REPORT"J.... 24
Page
vii
Page
Note, Discriminatory Merit Systems; A Case Study of
the Supervisory Examinations Administered by the
New York Board of Examiners, 6 COLUM. J. L. &
SOCIAL PROBS. 374 (1970) ......................... 5,22
ROGERS, 110 LIVINGSTON STREET (1968) ................. 24
SCHINNERER, A Report to the New York City Education
Department (1961) 24
STAHL, Public Personnel Administration (5th ed. 1962) 56
THORNDIKE, PERSONNEL SELECTION (1949) 54,56
THORNDIKE and HAGEN, MEASUREMENT AND EVALUATION IN
PSYCHOLOGY AND EDUCATION (1969) ................. 56
viii
UNITED STATES COURT OF APPEALS
FOR THE SECOND CIRCUIT
NO. 71-2021
BOSTON M. CHANCE, LOUIS C.
MERCADO, et al.,
Plaintiffs-Appellees,
-against-
THE BOARD OF EXAMINERS,
Defendant-Appellant,
and
THE BOARD OF EDUCATION OF THE
CITY OF NEW YORK, et al..
Defendants.
Appeal From An Order Of The United States District
Court For The Southern District Of New York
BRIEF FOR PLAINTIFFS-APPELLEES
Statement of the Issues Presented for Review
1. Whether the district court's finding that the supervisory
examination system administered by defendants had a significant
and substantial discriminatory impact upon blacks and Puerto Ricans
must be upheld as not clearly erroneous?
2. Whether the district court was correct in ruling that
examinations which have such a discriminatory impact violate the
Equal Protection Clause of the Fourteenth Amendment if they
cannot be shown to be job-related?
3. Whether the district court's findings that the supervisory
examinations administered by defendants had not been shown to
be and apparently were not job-related must be upheld as not
clearly erroneous?
4. Whether the district court's grant of preliminary relief
must be upheld as not constituting an abuse of discretion in light
of its findings with respect to irreparable injury, the balance
of hardships and the probability of plaintiffs’ success on the
merits?
Statement of the Case
(1) The Proceedings Below
J JPlaintiffs brought this class action in September, 1970,
pursuant to 42 U.S.C. §§1981 and 1983, challenging the examinations
used to select principals and other supervisors in the City School
System. Plaintiffs charged that these examinations operated in
violation of the Equal Protection Clause of the Fourteenth Amendment
in that they discriminated against blacks and Puerto Ricans and
could not be justified as job-related. Named as defendants were
the Chancellor, Dr. Harvey B. Scribner, the Board of Education
and the Board of Examiners. Plaintiffs moved simultaneously for
--/ The named plaintiffs included numerous blacks and Puerto Ricans
who had failed previous supervisory examinations, who possessed all
the educational and experience qualifications established for various
different supervisory positions they sought, but who could not receive
regular appointments because they lacked the requisite license
(App. 139a, 143-45a).
2
a preliminary injunction preventing defendants from administering
the Elementary School Principals' Examination scheduled for
November 3, 1970 and from giving other supervisory examinations
in the future until they had been validated.
Plaintiffs documented their claim of discrimination with
undisputed evidence that blacks and Puerto Ricans were grossly
underrepresented in the supervisory ranks and on supervisory lists
as compared to the student population (App. 147-49a). in addition,
comparative statistics were introduced from other city school
systems not subject to New York's unique examination procedures,
including Rochester's, where candidates must satisfy New York
State certification requirements. These showed New York City to
have by far the lowest percentage of black and Puerto Rican
supervisors, and the lowest ratio of minority group supervisors
2 /
to minority group students (App. 197-98a). And finally the
statistics showed there was a much higher percentage of blacks
and Puerto Ricans in acting supervisory positions, where ability
to perform is the sole criterion of selection, than in licensed
positions (19% of acting principals as against 1% of licensed
principals). Plaintiffs also documented the culturally biased
nature of the examinations themselves, submitting expert evidence,
based on an analysis of the most important examinations given
_2y Thus Rochester has a black student population comparable to
New York's (30% as compared to 34%) but 11 times as many black principals.
3
by defendants in recent years, that they were likely to discrimi
nate unfairly against blacks and Puerto Ricans (App. 35-39a;
see also App. 149-50a).
Evidence was also introduced indicating that defendants'
supervisory examination procedures could not be justified as
job-related. Thus plaintiffs showed that defendants had never
performed a job analysis - concededly the first and essential step
in developing a job—related examination — for any supervisory
position. indeed the first step of the first job analysis plan
in the history of the New York School System had just been
undertaken in the form of a research proposal to discover the
qualities needed by an Elementary School Principal, the very
examination scheduled for November 3, 1970, which plaintiffs
. . . !/sought to enjoin. Plaintiffs also charged that the persons
responsible for developing and administering defendants'
examinations had no background or training in the testing field,
were selected on an ad hoc basis from the teaching and supervisory
ranks without having to satisfy any special qualifications or
selection procedures, or being provided any special training in
test construction and administration and, in addition, that they
were overwhelmingly of the white race. Finally plaintiffs showed
that no validity studies had ever been conducted indicating
-17 APP- Ex- 19; DuBois Affidavit, Ind. Doc. No. 44, paras.
7 and 8 . Documents not included in the Appendix or Appendix
Exhibit Volume will be referred to by their number on the Index to the Record on Appeal.
4
that defendants' examinations were in any way related to job
_4J
performance.
Plaintiffs also submitted evidence that authorities responsible
for selecting supervisory personnel had found no correlation between
possession of a license and ability to perform the job at issue,
that many licensed candidates had been found unqualified to serve
in the position for which they were licensed, and that in order to
select the most qualified candidates it had often been necessary to
appoint persons without the requisite licenses on an acting basis
(App. ll-34a).
After obtaining numerous extensions of time defendants finally
filed reply papers October 27, 1970, one week prior to the
scheduled November 3 Elementary School Principals' Examination.
On the basis of oral argument October 27 and 29, the court
decided it had not had sufficient opportunity to consider the
merits of the case before it and therefore refused to enjoin
the administration of the November 3 and other future examinations.
However, to maintain the status quo pending consideration of
plaintiffs' motion, the court granted a temporary restraining
order preventing the promulgation of new eligibility lists based
on such examinations (App. 101a).
4/ App. 15a, 12a, 28-29a, 150a, 37-39a. See generally Note, Dis
criminatory Merit System: A Case Study of the Supervisory Examinations
Administered by the New York Board of Examiners, 6 Colum. J. Law and
Social Problems 374, 377-88 (1970), Exhibit 13 to DuBois Affidavit,
Ind. Doc. No. 44.
5
The Board of Education took the position at this time that
the complaint raised issues of fact which should be determined at
trial, and that it would leave to the Board of Examiners the task
of defending their examination procedures (App. 99-100a). The City
School System's chief administrator, Chancellor Scribner, had
previously indicated that he would "prefer not to defend" against
the action since "[t]o do so would require that I both violate
my own professional beliefs and defend a system of personnel
selection and promotion which I no longer believe to be workable,"
a system inconsistent with the selection of "the most talented
supervisors; the most able principals and other administrators who
possess the highest level of leadership qualities possible. . . . "
(App. Ex. 18, p. 4). Neither the Board of Education nor the
Chancellor has actively participated in the defense of this action
to date.
The Examiners argued in their reply papers that plaintiffs
had not made out a case of discriminatory impact because they
had not submitted "the only meaningful statistic" — to-wit,
"̂ .comparison of the pass-fail ratio of black and Puerto Rican
applicants." They argued further that
"[i]f such an evaluation were undertaken,
it would demonstrate clearly that eligible
black or Puerto Rican applicants have per
formed comparably with white applicants on examinations." 5 /
_5_/ (Affidavit of defendant Examiner Jay E. Greene App. 87-88a)
(emphasis added). in addition, Mr. Greene cited a few examples of
a few examinations in which "It is believed that" blacks and Puerto
Ricans performed comparable with whites (App. 88-90a).
6
The Examiners defended the validity of their examinations
at this time on the basis of a number of alleged research and
validity studies which they did not submit. Thus defendant
Examiner Rockowitz claimed there were numerous research studies
which were designed "to determine the validity" and which "confirmed
the validity" of the examinations (App. 63a); that there had been
at least three "empirical validity" studies, and that their
results "indicate substantial positive correlations between the
purposes of the examinations involved and the results obtained. . .
(ftPP• Ex. 10, pp. 3—4) (emphasis added). Specifically, he relied
on the results of an alleged predictive validity study of
elementary school principals' examinations as showing that
principals identified as the best performers on the job "score
higher on the . . . examinations than do a random selected group
of principals" (App. Ex. 10, p. 4). And finally he alleged that
the Examiners had conducted 22 "validity studies" and had
published three volumes relating to research (App. Ex. 10, p. 5).
The Examiners also relied on the opinions of four experts whose
opinions were based on statements made to them by Board representa
tives describing examination procedures and on the research
studies cited by Dr. Rockowitz, together with other documents
submitted to them. These experts claimed only that on the basis
of what they had seen and been told the Board appeared to have
made certain efforts to design content-valid examinations.
term
_6_/ The/predictive validity, used by the court below, is some
times referred to as "empirical" or "criterion related" validity.
A brief discussion of the meaning of predictive and content validity,
and significance in showing job-relatedness, appears atpp • Orr j o ; m r r a .
- 7 -
1
In response plaintiffs submitted extensive additional
evidence that defendants' examinations could not be shown
to be job-related. Recognized experts, including Dr. Enneis,
staff psychologist in the Office of Research of the U. S.
Equal Employment Opportunity Commission, concluded
that the documents entitled "Duties of the Position," which
were the only job descriptions used by the Examiners in con
structing examinations, did not constitute job analyses and
were an inadequate basis for developing a job-related selection
system (App. 108-09a, 117a). They also concluded that the
Examiners' experts had not conducted the kind of analysis and
investigation essential to support a claim of content validity
(App. 112-114a); and that in any event predictive validity
studies were both feasible and, indeed, were essential to
establish job-relatedness for the type of examinations administered
by defendants (App. lll-12a, 114a, 116a). Plaintiffs intro
duced additional evidence documenting previous charges that no
job analysis had ever been conducted, and that the assistant
examiners who designed and administered examinations had no
special background or training (App. 129a, 131-32a, App. Ex.
14) .
At a hearing November 19, 1970, the court indicated that
comparative statistics as to actual pass rates of different
ethnic groups were essential to assess the discriminatory impact
of defendants' examinations. Since plaintiffs had no access
to such information and since defendants claimed they had no
8
racial records, the court ordered the parties to develop a
survey procedure to determine comparative pass rates on a
significant sample of examinations.
Plaintiffs asked that defendants be ordered to produce
statistics on the pool of eligibles, as well as on applicants,
so as to document the charge that defendants' discriminatory
examination system had a chilling effect, discouraging blacks
and Puerto Ricans from even applying. Defendants had claimed
that no such statistics were available, and the court refused
to order them produced on the grounds that the essential issue
in determining discriminatory impact was the effect of the
examinations upon actual applicants (Ind. Doc. No. 25, TR 10-13, 16).
Elaborate survey procedures were worked out by the parties
and the court and incorporated in four different court orders.
The survey covered the most important supervisory examinations
given in the past seven years and all supervisory examinations
given in the past three years. Over 6000 applicants were in
volved. The survey was conducted by the Board of Examiners and
the Board of Education with the assistance of the National Opinion
Research Center, and took several months to complete. Following
the submission by both sides of expert affidavits (Ind. Doc. Nos.
61-63, 66) and briefs (Ind. Doc. Nos. 60, 64) directed to the
relevance of the statistical tabulations, an evidentiary hearing
was held May 21, 1971 (Ind. Doc. No. 26), to clarify apparent
conflict between the parties' experts. At this hearing plaintiffs
presented essentially uncontradicted expert evidence that the
7/ App. 189a, 231a n. 14; Ind. Doc. Nos. 11, 15, 23.
9
statistics revealed that defendants' examinations had a significant
and substantial discriminatory impact upon blacks and Puerto Ricans.
At the conclusion of the hearing the court afforded both
sides an opportunity to provide further evidence by affidavit
or oral proof respecting the issue of job-relatedness (TR 110,
Ind. Doc. No. 26).
Plaintiffs submitted all the Examiners' alleged research
and validity studies which were obtained upon court order pursuant
to a discovery request, Ind. Doc. No. 17. Plaintiffs also submitted
an extensive expert analysis of these studies in the form of an
affidavit submitted by Dr. Richard Barrett (App. 155-64a, App.
Ex. No. 15), a recognized authority on employment testing and
discrimination (App. 35-36a). Dr. Barrett stated that he had
examined all the alleged research and validity studies relied
upon by Dr. Rockowitz in his original affidavit for his claim
that the examinations were job-related. Dr. Barrett concluded
that no study supported a claim of predictive validity and no
study related to a claim of content validity. He stated further
that the only study even relating to validity, entitled Analysis
of Examination for Principal of Elementary Schools (Exhibit 1 to
Barrett Affidavit, Ind. Doc. No. 68), indicated that there was in
fact no relationship between the examinations in question and
job performance (App. 159-61a). The remaining research studies
consisted solely of studies dealing with the internal structure
of tests, the research proposal for the development of a job
analysis for the Elementary School Principal position referred
10
to supra, p. 4, descriptions of teacher selection procedures
in various school systems in the United States, and a report on
a conference held by the Board of Examiners, also involving
_SJteacher selection procedures (App. 161-62a).
In addition, Dr. Barrett stated that he had examined all
the documents relied on by the Examiners' four testing experts
as the basis for their opinions concerning the Examiners' pro-
cedures. Dr. Barrett concluded that mere examination of these
documents could not support a claim for content validity (App.
l62-63a), and that such a claim could be justified only by the
kind of study which it was clear from their affidavits the
Examiners' experts did not conduct — that is, a thorough
analysis of the various supervisory jobs at issue, establishment
of success criteria and determination of whether the tests at
issue adequately represented the most important aspects of the
related jobs (App. 157-58a) (see also 112-14a). Indeed Dr.
Barrett noted that the Examiners' experts had not even claimed
that the examinations were in fact content valid but only that
it -appeared that the Examiners attempted to design content valid
examinations (App. 157a).
8 / Each of these documents are described in the appendix to
Dr. Barrett's affidavit which appears at App. Ex. 15. The
documents themselves were filed as Exhibits 2-8 to Barrett
affidavit, Ind. Doc. No. 68.
9_/ (App. 158a) (See supra p. 7 ). These documents were also
filed as Exhibits to the Barrett affidavit , Ind. Doc. No. 68.
In response the Examiners, who had previously declined
the court s offer of an opportunity to submit additional evidence
by affidavit or oral testimony (Ind. Doc. No. 26, TR 105, 109, 110),
chose to submit only another affidavit from defendant Examiner
Rockowitz which added nothing to his previous claims (App. 165-77a).
(2) The District Court's Decision
On the basis of a voluminous record, briefly summarized 10/
above, Judge Walter R. Mansfield, who had presided over every
aspect of the case from its initiation, issued his opinion July 14,
1971 (App. 179-235a), finding that the Board's examinations had the
"effect of discriminating significantly and substantially against
qualified black and Puerto Rican applicants." (App. 201a; see also
190-91a, 221a).
The court's analysis of the comparative pass rates revealed
by the court-ordered examination Survey was based on the affidavits
submitted by statisticians for both sides and the evidentiary
11/hearing held May 21, 1971 (App. 190-201a). The court found that
analysis of the aggregate data for the entire series of examinations
showed whites passing at almost one and one—half times the rate of
blacks and Puerto Ricans (App. 190a, 194-96a). The court noted
fur"ther that whites had passed the examination for Assistant Principal,
_10/ The court noted that in reaching its decision it had had the
benefit of "a plethora of lengthy affidavits and exhibits, a hearing
at.which oral testimony was taken, a series of arguments, and extensive
briefing of the law and facts by the parties," as well as briefs bythree amicus parties (App. 183a).
11/ To the extent that the parties' statisticians differed, the court
noted that it was more persuaded by the testimony of plaintiffs'
statistician after "reviewing their testimony and appraising them as witnesses" (App. 194a).
12
Junior High School at almost double the rate of blacks and
Puerto Ricans and had passed the examination for Assistant
Principal, Day Elementary School at a rate one-third greater than
blacks and Puerto Ricans, and that these examinations were of
particular significance both because of their size and "because
the assistant principalship has traditionally been the route to and
prerequisite for the most important supervisory position, Principal"
12/(App. 191-92a). Finally, the court concluded, on the basis of
the expert evidence, that the fact that whites passed at a higher
rate than blacks and Puerto Ricans in 25 of the 32 examinations
subject to comparison, was proof of the discriminatory impact
13/
of the entire series (App. 193-94a).
However, the court did not rely solely on the statistical
disparities revealed by the Survey since it was clear from the
evidence that they represented a gross underestimate of the dis
criminatory impact of defendants' supervisory examination system.
12/ The court noted that of the 50 examinations covered by the
survey some were taken by very few people and because of the smallness
of the samples the results of each, analyzed individually, could not
be accorded much significance. Thus 41 of the 50 examinations were
taken by only 83 (or 10.1%) of the total number of black and Puerto
Rican candidates (App. 196-97a). On all nine examinations taken by
10 or more black and Puerto Rican candidates, whites passed at a
substantially higher rate (App. 197a).
13/ The results of Dr. Cohen's statistical analysis are summarized
at App. Ex. 26.
It is worth noting that there was no significant conflict
between the two statisticians at the evidentiary hearing below.
Dr. Jaspen, the Examiners' statistician, conceded that Dr. Cohen's
analysis of the aggregate data was correct (Ind. Doc. No. 26,
TR 66-67); that it was proper to subject the aggregate data to
13
Thus the court found that:
"The fact that the process involves a series
of examinations and that to reach the top
one must pass several examinations at different
times in his or her career serves to magnify
the statistical differences between the white
and non-white pass-fail rates." (App. 192a)
Thus, for example, if any given examination screened blacks and
Puerto Ricans out at approximately twice the rate of whites, two
successive examinations would screen them out at four times the
rate of whites (App. 192-93a).
On the issue of whether examinations with such an impact could
be justified the court held:
"The Constitution does not require that minority
group candidates be licensed as supervisors in
the same proportion as white candidates. The
goal of the examination procedures should be to
provide the best qualified supervisors, regard
less of their race, and if the examinations appear
reasonably constructive to measure knowledge,
skills and abilities essential to a particular
position, they should not be nullified because
of a de facto discriminatory impact."
[Emphasis added] (App. 201a)
The court concluded, however, that the Examiners' procedures could
not be justified as being reasonably related to job performance.
On the basis of extensive evidence and exhaustive briefs discussing
the law and authorities on employment testing,the court found that
for a test to be considered job-related on the basis of its
13/ (Continued)
analysis to determine the overall impact of the supervisory
examinations on candidates (id. at TR 68-6 9) ; and that it was
proper to compare the number of examinations whites passed at a
higher rate with the number of examinations blacks and Puerto
Ricans passed at a higher rate, by the "binomial" or "sign" test,
in assessing the impact of the series of examinations (id. at
TR 69-71) (apart from the problem of overlap discussed Toy the court
at App. 195-96a).
14
“.£2nt.?nt validity," it was essential to first perform an
adequate job analysis:
"Such an analysis requires a study to be made
of the duties of the job, of the performance
by those already occupying it, and of the
elements, aspects and characteristics that
make for successful performance. Questions
are then formulated, selective procedures
established, and criteria prepared for
examiners that should elicit information
enabling them to measure these characteristics,
skills and proficiency in a candidate and
determine his capacity to do the job satisfactorily. (App. 206a)
The court noted that the Examiners' own expert, Dr. Thorndike,
had stated that "predictive validity,'1 which depends upon a
showing of a correlation between test scores and job performance,
was "most desirable. . . . to show that the test is in fact
effective in discriminating between those who are and those who
are not successful in a particular job," and that anything
short of that must be recognized as a "stop-gap" (App. 206a).
Although plaintiffs had contended that the Board's examinations
could not be considered job-related without a showing of
p redictive; validity, the court did not have to reach this issue.
It found that even accepting the Examiners' position that use
of procedures designed to ensure content validity was sufficient
(App. 212-13a), the Board had not "achieved the goals of con
structing examination procedures that are truly job-related"
since:
Despite its professed aims the Board has not
m practice taken sufficient steps to insure
that its examinations will be valid as to
content, much less to predictiveness."
(App. 213a)
15
claim that they relied on outside experts and lay persons in
determining success criteria was not supported by the evidence
(App. 213-14a), that the Chancellor had found the examinations a
barrier to selection of the most qualified supervisors (App. 214-15a),
and that the Examiners' position was not supported by the research
reports they had relied on ". . .as demonstrating the content
validity of . . . supervisory examinations." (App. 215-16a).
The court found that the only report even relating to validity was
a pilot predictive validity study which "showed that there was
little or no correlation between success on the tests and job
success." (App. 216a). The court concluded that the Board had
failed to achieve the goal of content validity, noting that this
conclusion, "which is based upon our appraisal of affidavits of
experts furnished by the parties, is confirmed by our own study
14/
of some of the examinations. . . . "
The court concluded that preliminary relief should be
granted since there was "a strong likelihood that plaintiffs
will prevail on the merits at trial,""the balance of hardships
tips decidedly in favor of plaintiffs, and, pending final determi
nation of the merits, the effect of preliminary relief would be
to preserve the status quo until the issues are resolved," and
preliminary relief would not harm the public (App. 223-25a).
In support of this finding the court noted that the Examiners'
14/ App. 217a. The court noted that defendants' examinations
typically contained both short-answer, multiple-choice tests and
essay tests, both of which appeared "aimed at testing the candidate's
ability to memorize rather than the qualities normally associated
with a school administrator" (App. 217a), and that a leading
training manual for such examinations emphasized the use of mnemonic
devices (App. 217-19a). See 232-25a, n. 23-24 for examples.
16
(3) The Preliminary Injunction
At defendants’ request and over plaintiffs’ vigorous
objection, the court delayed entry of the preliminary injunction
because counsel for defendants had alleged they were unable to
consult with their clients during the summer.^
The Examiners took advantage of this time to develop
partial statistics and prepare new affidavits relating to the
November 3, 1970 Elementary School Principals' Examination and
xn September, 1971, urged the court to consider this new evi
dence and to permit licenses to be issued and regular appoint
ments made on the basis of that examination. Despite the fact
that the motion for a preliminary injunction and the court's
July 14 opinion were focused largely on that very examination, and
the record was closed, the court nonetheless carefully considered
the Examiners' papers, rejecting their relevance for reasons
set out in detail in the memorandum opinion supporting its
preliminary injunction (Ag£. 251-55a). After considering a
variety of proposed injunctive orders submitted by the parties,
together with supporting affidavits, memoranda and other papers,
the court entered its preliminary injunction on September 20,
1971 (A££. 2 57-59a), one year after suit was filed, incorporating
certain of plaintiffs' suggestions, but permitting any party to
apply for a modification of the order to permit the institution
of new examination procedures (App. 259a).
.15/ The court meanwhile entered an interim inj
defendants from conducting further examinations
eligibility lists, and using outstanding lists
appointments (App. 237a).
unction restraining
, publishing new
to make permanent
17
(4) Subsequent Developments
At the time the court entered its injunction the Chancellor
had submitted a resolution proposing the institution of an
interim system for the selection of acting supervisors to
operate pending final determination of the instant action
(App. Ex. 27), which the court below noted as bolstering its
previous determination that the relief endorsed would in no
way harm the public or the administration of New York City
schools (App. 255a). A slightly modified version of this pro
posal was adopted by the Board of Education on October 6, 1971,
and is presently being implemented. It contains provisions
for the development of "written procedures governing the
selection and assignment of acting supervisory personnel,"
"the description . . . of clear performance objectives for the
position to be filled," the development of "performance criteria,"
the "periodic evaluation of on-the-job performance of acting
supervisory personnel," and an "Advisory Council on the
Selection of Acting Supervisory Personnel" to ensure that
aPpointments are made in consonance with merit and fitness.
The resolution also established new eligibility requirements
for acting supervisory positions. 16/
On November 9, 1971, the Examiners submitted to plaintiffs
a proposal for a new supervisory examination system (attached
hereto as App. A, pp. 2-13a infra). This proposal was sub
mitted as the basis for an application to the court for modi
fication of the preliminary injunction (pp. l-2a, infra).
15/ A * •Acting supervisors must meet eligibility requirements for
2 ; ro ^ a ? r ; ? ^ p^ - j r rvisory exa” ination -
18
Designed to improve the "compatibility" of the examinations
with a decentralized system (p. 3a, infra), this proposal
makes clear in describing the steps that would be taken in
developing job analyses, job-related examinations and
validation procedures that previous procedures were inadequate
to ensure job-relatedness. Thus for example, the proposal
states that the Board will obtain
" . . . in addition to statements of duties of
the position, appropriate skills, qualities and
behaviors, in other words, specific job analyses
for each position for which it is required to con
duct examinations." (p. 4a, infra)
The proposal describes the manner in which job related examinations
will be developed, noting that
"[t]he language mastery element in written tests
will be deemphasized to take its place alongside
such data as are obtained through other tests
which reveal essential elements of administrative
ability, supervisory ability, attitude toward the
learning process for the varying student popu
lations of the city, human relations skills as
the candidate may offer, e.g., knowledge of spoken
Spanish, knowledge of Afro-American and Hispanic
culture, etc." (p. 5a, infra)
The proposal suggests that instead of eliminating candidates
by rigid cut-off scores, as in the past, the Examiners would
instead provide an "assessment profile" for each candidate,
thus enabling such community school board to evaluate the
various candidates in the light of its own particular needs;
only those candidates who could not satisfy a level of mini
mum competence would be disqualified by the Board's examinations
(pp. 6-8a, infra). Finally, the proposal provides for the
predictive validation of the new examination system (pp.ll-12a,
infra).
19
Subsequently, the Educational Testing Service of Princeton,
New Jersey (E.T.S.), submitted to plaintiffs and the Examiners
a draft proposal for the development by E.T.S. of a truly non-
discriminatory and job-related examination system. This pro
posal (attached hereto as App. B, pp. 14-24a,infra) reveals
beyond any doubt that the Examiners' previous procedures were
wholly inconsistent with professional test development standards
and incapable of producing job-related examinations. The pro
posal also makes clear that a job-related examination system
can be developed, describing in some detail the steps that
should be taken to develop job analyses, success criteria, and
test specifications. Finally, the E.T.S. proposal shows that
empirical validation, which rests on a demonstrated correlation
between test scores and job performance, is not only feasible
but accepted by professionals in test technology as a neces
sary preliminary to use of an examination (pp. 19a, 22-23a,
infra). It is also significant that E.T.S. starts with the
assumption that a job-related and non-discriminatory examination
would not reject members of different ethnic groups out of
proportion to their numbers in the applicant group (p. 16a,
infra).
20
ARGUMENT
Introduction
In reviewing the decision below it is important to realize
that the trial judge who presided over every phase of the case
during the entire 12 months of proceedings, acted on the basis
of an exhaustive factual record, which included literally
dozens of affidavits, hundreds of pages of exhibits, and an
evidentiary hearing. The court also relied upon numerous
briefs, submitted at various stages of the case by the parties
and three amici curiae, and several oral arguments. Through
out the proceedings the court proceeded with extreme caution,
providing only such interim relief as seemed essential, and
refusing to make any assumptions as to the discriminatory
impact of defendants’ examination system in the absence of a
comprehensive summary of examination results. At every stage
the court offered the parties full opportunity to present
evidence and argument. While the Examiners now object to
certain of the court's factual conclusions they could not
and did not object at any time that the court prevented them
from presenting their case in whatever form they chose.
It is also important to realize the context in which the
legal issues involved in this case appear. There has been
increasing recognition in recent years that New York City's
schools are in a state of crisis, that ghetto schools in par
ticular are failing to educate their students, and that black
21
and Puerto Rican children not only do badly as compared to
whites but do worse the longer they stay in these schools.-^
It is in large part out of this recognition that the movement
for decentralization grew, in the hope that schools would
thereby become more responsive to the needs of their com
munities and more able to serve their particular student
bodies. It is particularly ironic in this context that
defendants should maintain an examination system which is
not job-related and which in no way accounts for the fact
that different qualifications might be required for positions
in different schools and communities - a system which locks
out of the supervisory ranks many persons who are particularly
qualified to serve the students in the ghetto schools which
are at present failing their educational task.-̂ "^
17/ „See Note, supra n. 4, at 387-88 and n. 115; Council of
Supervisory Associations v. Board of Education, 23 N.Y.2d 458,
463, 297 N.Y.S.2d 547, 551, 245 N.E.2d 204, 207 (1969).
18/ Two recent cases have upheld administrative decisions to
by-pass normal civil service lists for school supervisory
positions, largely in recognition of the fact that traditional
civil service procedures have failed to select the very people
most needed to relate to ghetto communities and thereby serve
the schools which are most in need of improvement. In Council
of Supervisory Associations v. Board of Education, 23 N.Y.2d
458, 297 N.Y.S.2d 547, 245 N.E.2d 204 (1969), the New York
Court of Appeals upheld the Board of Education's decision to
create the new position of Demonstration Elementary School
Principal, and to make acting appointments to said position
pending development of a relevant examination, thereby by
passing the Examiners' Elementary School Principal list.
The court's decision was premised in part on its recognition,
in light of New York's educational crisis, of the importance
of selecting principals who could relate to their communities
and its finding that the traditional examination system had
failed to select such principals. See also Porcelli v. Titus,
302 F. Supp. 726 (D. N.J. 1969), aff'd 431 F. 2d 1254 (3d Cir.. 1970). -----
22
The evidence below revealed the gross disparity between
the numbers of blacks and Puerto Ricans on the supervisory
staff and in the student body (Ap£. 187-88a). Thus blacks
and Puerto Ricans constituted over half of the student body
but only one per cent of the principals. Whatever factors
may have contributed to this in the past it is clear that, with
decentralization and with revised eligibility requirements,
the Examiners' procedures had become the ma]or barrier to the
employment of minority supervisors.— ^ The evidence below
12/ Even if the examinations themselves did not disqualify
blacks and Puerto Ricans at such significantly disproportionate
rates, they would nonetheless, given the exclusionary effect
of the system as a whole, constitute unconstitutional barriers
to defendants' affirmative obligation to license and appoint
qualified black and Puerto Rican supervisors. The Survey
illustrated the manner in which defendants' system operated
as a whole to effectively prevent the appointment of any sig
nificant numbers of blacks and Puerto Ricans to the super
visory ranks, because of the combination of such factors as
low pass rates, eligibility requirements and exhaustion of
lists. See Plaintiffs' Memorandum on Relevance of Statistical
Tabulations, Ind. Doc. No. 60, pp. 10-12. A system of selection
which operates to perpetuate the gross underrepresentation of
blacks and Puerto Ricans on that staff and to prevent the
appointment of blacks and Puerto Ricans who might be qualified,
violates the Equal Protection Clause of the Federal Constitution.
See generally Porcelli v. Titus. 431 F.2d 1254, 1257-58 (3d
Cxr. 1970), affirming 302 F. Supp. 726 (D. N.J. 1969), holding
that where there was a gross disparity between the percen
tage of blacks in the student population and their percen
tage on the supervisory staff, the school board had an affir
mative obligation under the Federal Constitution to correct
that disparity. See also Jackson v. Wheatley School District.
430 F.2d 1359, 63 (8th Cir. 1970); Armstead v. Starkville
Municipal Separate School District. 325 F. Supp. 560, 569
(N.D. Miss. 1971), where the courts found disparities between
the percentage of black students and the percentage of black
faculty members evidence of racial discrimination. In North
Carolina Board of Education v. Swann, 402 U.S. 43 (1971), the
Supreme Court made it clear that any law, regulation or of
ficial policy which prevented boards of education from taking
23
further revealed that the Examiners' procedures were blocking
the appointment of minority and other candidates not on the
basis of "merit and fitness" but, rather, on the basis of
their inability to answer such questions as "[W]ho killed
Cock Robin?" (App. 232a, n. 23).
The Board's procedures have for decades been subjected
to severe criticism by every body which has investigated it
because, far from fulfilling their purported goal of selecting
the best qualified supervisors, they were in fact locking such
persons out and perpetuating a sterile, bureaucratic, insider
system.20/ But despite this criticism and series of recom
mendations for reform, no significant change in the Board's
procedures took place. It is only as a result of the decision
19/ (Cont'd)
affirmative action to correct the kind of gross racial under
representation exhibited in the instant case must be declared
unconstitutional. See generally Fiss, A Theory of Fair Employ
ment Laws, 38 U. CHI. L. REV. 235, 270 n. 42 (1971).
20/ See Mayor's Committee on Management Survey, Administrative
Management of the School System of New York City (1951) (the
"Strayer and Yavner Report"); Schinnerer, A Report to the New
York City Education Department (New York, 1961); Cresap,
McCormick and Paget, Summary Report of Assignments Conducted
for the New York City Board of Education (1962); GRIFFITH REPORT,
TEACHER MOBILITY IN NEW YORK CITY: A STUDY OF RECRUITMENT,
SELECTION, APPOINTMENT, AND PROMOTION OF TEACHERS IN THE NEW
YORK CITY PUBLIC SCHOOLS 162-230 (1963); Center for Field
Research and School Services, New York University, A Report
of Recommendations in the Recruitment, Selection, Appointment
and Promotion of Teachers in the New York City Public Schools
(1966); BUNDY REPORT, MAYOR'S ADVISORY PANEL ON DECENTRALI
ZATION OF THE NEW YORK CITY SCHOOLS, RECONNECTION FOR LEARNING
110, n. 28 (1967); Rogers, 110 LIVINGSTON STREET 285-97 (1968).
In addition, Boards of Education have, in recent years, urged
the legislature to abolish the Board of Examiners. And most
recently the New York City Commission on Human Rights issued a
24
of the court below that there now appears to be some hope for
the development of a selection system which will truly allow
for the appointment of those supervisors best qualified to
serve the schools and children of this City (see pp. 18-20, supra).
POINT I
THE DISTRICT COURT'S FINDING THAT
DEFENDANTS' EXAMINATION SYSTEM HAD
A SIGNIFICANT AND SUBSTANTIAL DIS
CRIMINATORY IMPACT WAS CLEARLY
SUPPORTED BY THE RECORD
Throughout their brief the Examiners claim that the court
below based its decision on statistical rather than legal
analysis, and found a prima facie case of discrimination made
out on the basis of findings of mere "statistical significance"
and departures from "perfect equality" in the treatment of
20/ (Cont'd)
thorough report on the schools' selection system, based on
public hearings held Jan. 25-29, 1971 and extensive documen
tation, recommending that the Board of Examiners be dis
continued on the grounds that its present examination system
locked out qualified minority and other candidates, and was
apparently not job-related. The Commission recommended further
that New York City rely on state certification for initial
screening (as does virtually every city school district in
the country) and on community school boards for actual
selection based on "criteria and procedures geared to the
need of individual boards." Equal Employment Opportunity and
the New York City Public Schools, An Analysis and Recommen
dations Based on Public Hearings Held January 25-29, 1971 by
the City Commission on Human Rights, (pp. xv-xvi). The Com
mission specifically found that the lists of job duties used
to prepare examinations were "inadequate bases for sound
test construction" (p. xx) and that the Board of Examiners’
resources were inadequate to the task of designing and adminis
tering tests and assessing their validity (p. xxi).
25
members of different ethnic groups. (See Brief pp. 10, 15-17,
21-24) This is simply not true. The court's decision was
clearly based not on a test of mere statistical significance
but, rather, on findings that defendants' examinations had a
significant and substantial discriminatory impact, resulting
in "gross" differences in the treatment of different ethnic
groups (App. 201a, 191a, 221a; see also 190a).
The court's finding of discriminatory impact was based
essentially on two factors. The court relied first on the
results of the survey, which showed that the entire series
of examinations had a significant and substantial discrimi
natory impact on black and Puerto Rican applicants and also
showed gross disparities in pass rates on the biggest and
most important examinations (pp. 12-13, supra; App. 189-94a*
App. Ex. 26). But secondly, the court concluded that the
disparities revealed by the survey constituted a gross under
estimate of the discriminatory impact of the examination
system because, as the evidence revealed and the court found,
during the period covered by the survey the promotional process
involved a series of examinations and "to reach the top one
must pass several examinations" (App. 192a). The court found
21/ Indeed, the reason the court apparently felt called upon
to discuss the statistical analysis of the data in such detail
is because the Examiners had rested their case largely on an
attempt to attack the statistical significance of the survey
results. See Dr. Jaspen's affidavit, Ind. Doc. No. 66 and
May 21 hearing TR, Ind. Doc. No. 26.
26
that the net effect of this system was to screen minority
candidates out at rates far in excess of those indicated by
the survey results (App. 192-93a). ii/
22/
The Examiners miss the point in their brief (pp. 18-19)
since they assume erroneously that the example the court gives
to illustrate the manner in which this multiplier effect
worked was intended as some kind of factual future prediction.
The Examiners introduce in their brief at p. 19 partial
and misleading pass-fail data on the results of the November
1970 Elementary School Principals' Examination. These data,
produced after the court's decision, when plaintiffs had no*
opportunity to present evidence, were rejected below in the
court s memorandum opinion supporting the preliminary injunction
(Ap£. 251-55a), on the grounds that they were too incomplete
to be meaningful (covering only 60% of the candidates), that
they revealed that the pass rates for the identified candidates
could not be assumed to be typical of the pass rates for those
not identified, that plaintiffs had had no opportunity for
cross-examination, and that plaintiffs were entitled to
discovery to determine all relevant facts relating to this examination (App. 253-54a).
The fact that the Board of Education has recently re
vised certain eligibility requirements is irrelevant since
during the period of the survey the previous requirements
were in effect. Moreover, despite the revisions most candi
dates must still pass through a succession of tests starting
with the teacher's examination. Finally and most important,
to the extent that the Board of Education does reduce the
number of examinations through which candidates must pass as
they climb the promotional ladder, it is likely that the dis
criminatory impact of each particular examination (assuming
that their nature remains unchanged) will increase. This is
because candidates who have survived screening by prior tests
are relatively homogeneous in test-taking ability and un
screened groups are likely to show greater differences between
different ethnic groups (Dr. Cohen's affidavit, ver. 5/6/71,Ind. Doc. No. 62, para. 7).
It is worth noting in this connection that the survey
results grossly underestimate the discriminatory impact of
defendants examination system in an even more significant
way, not taken into account by the court below. Virtually
all candidates covered by the survey had been pre-screened
27
In addition, the court relied on the fact that cities
which did not use an examination system like New York1s had much
higher percentages of blacks and Puerto Ricans on their super
visory staffs (App. 197-98a; 222a; see also p. 3. supra)
The Examiners also quarrel with the court's analysis of
the survey results in numerous respects. In considering their
arguments it is important to realize that the survey procedure
was developed by the parties and the court as the best method
of assessing the impact of defendants' examinations; that the
parties' statisticians differed on no significant issues (see
n. 13 , supra); that defendants had ample opportunity to present
any evidence they considered relevant below; and that the
court made its findings on the basis of exhaustive evidence
and briefing.
22/ (Cont'd)
by the Board's teacher examinations and it is likely that it
was at this stage, where the applicant group was unscreened
by any previous test, that the discriminatory impact was
greatest. what is extraordinary is that the Board has managed
to design examinations which discriminate even against blacks
and Puerto Ricans who have proven themselves to be good test
t.c l.K 6 ITS •
23/
, fvTh! Examiners claim that these disparities are explained
bT th^-faCt that New York city's education and experience eligibiiity requirements were higher than those in "many other cities (Brief pp. 20-21), but there is no evidence in the
record that this is true with respect to any specific city named by the court. J
28
The Examiners contend now, for the first time, that the
comparative pass rates revealed by the survey are irrelevant,
on the grounds that the minority applicant group allegedly
constitutes a disproportionately large percentage of the pool
of eligibles and is allegedly less qualified than the white
applicant group (Brief pp. 11-15). This argument depends on
a number of wholly unsupported and unwarranted assumptions:
first, that the teacher population approximates the pool of
eligibles; second, that the ethnic groups within this popu
lation are equivalent in ability; and third, that among those
®^icfihle it is the most qualified who apply for promotional
examinations.
Presentation of this issue now is, however, untimely
since it was not raised below where there would have been
ample opportunity to develop and present relevant evidence.
Instead the Examiners insisted below, and the court conceded,
that the best evidence of the examinations' discriminatory
impact would be comparative pass rates of actual applicants.
(PP-4, 9, supra) Thus the Examiners took the position below
that "the only meaningful statistic would be a comparison of
the pass-fail ratio of black and Puerto Rican applicants"
(ftPP- 88a). At no point in the proceedings below did they
indicate that these were not the most relevant data. Before
the survey their claim was that blacks and Puerto Ricans
passed at the same rate as whites. After the survey their
claim was that there was no significant difference in the
pass rates revealed. it is only now that the court below has
29
found that the differences are in fact significant and sub
stantial that the Examiners claim that the entire survey and
all the proceedings below related to it were an exercise in
futility because the focus should have been on the pool of
eligibles rather than the applicants. a e Examiners can_
not now attempt to justify the disparities in pass rates
revealed by the survey on the basis of assumptions - for
which there is no support in the record - that the minority
applicants were less qualified than the white applicants.^
Moreover, there is no reason to credit the Examiners'
unsupported assumptions that the minority applicants were less
qualified. First there is no reason to assume that the teacher
24/
bSl?W P o n t i f f s attempted to obtain statis-ics on the pool of eligibles defendants refused to provide them. See p. 9 , supra. ^
25/
*£deed below there was uncontradicted expert testimony
he analysis of the survey results was relevant "for the
population at issue, not merely the sample under study."
(Hearing May 21, Ind. Doc. No. 26, TR 13).
30
population approximates the pool of eligibles.^ Secondly
different ethnic groups in the teacher population are not
necessarily equally qualified. Since only highly motivated
minority persons are likely to try to break into a system
that has traditionally been closed to them, the minority group
may be particularly well-qualified. Finally, there is no
reason to assume that in this school system promotional
examinations will attract the most qualified candidates
from every ethnic group. The most talented teachers may well
decide to leave the school system after a few years. This
would likely be particularly true of whites who have more
opportunities open to them. The less talented and more
system-oriented whites, on the other hand, might well tend to
seek supervisory positions. Disproportionate numbers of highly
qualified blacks and Puerto Ricans might, however,
apply for promotional examinations because it is only in recent
26/
Substitute teaching and acting supervisory experience is
often an alternative to licensed experience and blacks and
Puerto Ricans represent a much larger percentage of substitute
teachers and acting supervisors. Similarly, experience as a
teacher anywhere in the United States under a valid license
satisfies the requirement of previous teaching experience.
And since the qualifications for each position are different,
there is simply no way of estimating the pool of eligibles
for the various examinations. See generally Exhibits 1-9 of
ver- 1-0/19/71. fid. DOC. No. 44. for examplesof qualifications. r
31
years that opportunities for appointment have begun to open
up and, therefore, that these examinations represent a real
opportunity for advancement. By contrast, the white applicant
group more likely includes large numbers of persons who were
previously and failed previous examinations.
While one could speculate endlessly as to the comparative
qualifications of the different ethnic applicant groups the
point is that the evidence below showed that the examinations
had a substantial discriminatory impact on black and Puerto
Rican applicants. The Examiners failed to provide any evi
dence justifying that impact, although they had ample oppor
tunity to document their hypothesis that the black and Puerto
Rican applicant group was less qualified than the white. in
the absence of evidence, such an hypothesis cannot be accepted
by this court, particularly since there is every reason to
believe that the black and Puerto Rican applicants constituted
a highly motivated and qualified group who persisted within a
system traditionally closed to them.
The Examiners' other objections to the court's analysis,
also raised here for the first time, are equally frivolous.
Thus they claim that the survey results cannot be used to
assess the discriminatory impact of defendants' examinations
in general, because examinations for different supervisory
positions were involved (Brief pp. 17-18). However, this
issue was-thoroughly explored at the evidentiary hearing
below and the court concluded, on the basis of testimony
by both statisticians, who evidenced no disagreement on this
32
issue, that valid conclusions as to the discriminatory nature
°f the examinations could be drawn even though discrete
examinations for different positions were involved. More
over the purpose of the survey designed by the parties and the
court was to assess the impact of the examinations as a whole ,
and the Examiners' only argument at that stage of the pro
ceedings was that fewer examinations should be included. The
Examiners also claim that the court erred in relying on com
parative pass rates for the entire group of applicants. They
contend that it should have considered as failing only those
who completed the entire examination process and not those
who failed because they withdrew at some earlier stage, either
before or after the written examination. But the court's use
of comparative pass rates was clearly reasonable and supported
by the expert evidence below. Moreover, both statisticians
testified that it made no difference whether the survey results
were analyzed by including or excluding the group of candidates
iZ/ See Hearing May 21, 1971, Ind. Doc. No. 26. Thus Dr. Cohentestified:
Q. Is analysis of aggregate test results relevant
to the sort of questions we have discussed when
there are different tests involved for different
kinds of supervisory positions and when the tests
involve different numbers of candidates?
A. Yes, they are quite adequate. (TR 9-10)
Q. Is it correct to use the binomial test where
you have a series of different examinations
for different positions?
A. Absolutely. . . . (TR 18)
See generally Dr. Cohen's testimony at TR 9-10, 13,14,18, 19-20,
22 and Dr. Jaspen's testimony at TR 68,70,71,81-82, 89.
In addition testimony by both statisticians completely ex
ploded the Examiners' argument below, repeated in its brief at
p. 17, that there was any relevance to the failure to find
statistically significant differences in pass rates in various
examinations with extremely small samples. See generally TR 22-32, 71-78.
33
who failed to complete the examination process.— '
POINT II
THE COURT BELOW CORRECTLY RULED THAT A PUBLIC
EMPLOYER VIOLATES THE EQUAL PROTECTION CLAUSE IN
CONDUCTING A PROMOTIONAL EXAMINATION SYSTEM WHICH
SYSTEMATICALLY AND SIGNIFICANTLY DISCRIMINATES
AGAINST ETHNIC MINORITY GROUPS WHERE THAT EXAMI
NATION SYSTEM CANNOT BE SHOWN TO HAVE ANY RELATION TO JOB PERFORMANCE
A ' A Prima Facie Case of Discrimination is Made Out
Where—Examinations Significantly and Systematically
Exclude Members of an Ethnic Minority Group
Regardless of Whether Any Subjective Intent to
Discriminate is ShownT
The E <aminers argue in their brief at pp. 21-26 that even
where there is evidence of grossly disparate treatment of racial
groups, additional evidence of purposeful discrimination is
essential to establish a claim under the Equal Protection Clause.
The danger of their argument, if accepted, is that it would
leave courts powerless to deal with the kinds of discrimination
problems that ethnic minorities will face increasingly as overt29/
forms of bias are outlawed. Moreover the interests of those
28/ see Hearing May 21, Ind. Doc. No. 26, TR pp. 10-11, 55. Thus
Dr. Jaspen, defendants' statistician, testified:
We analyzed this with pass versus fail plus did
not appear, plus withdrawn, and also just pass
versus fail, and we obtained completely consistent
results. . . . (TR 55)
.29/ Thus an Griggs v . Duke Power Co.. 420 F.2d 1225 (4th Cir. 1970),
rev_d, 401 U.S. 424 (1971), Judge Sobeloff noted in dissent:
The case presents the broad question of the use of
allegedly objective employment criteria resulting in the
denial to Negroes of jobs for which they are potentially
qualified. . . . On this issue hangs the vitality of
the employment provisions (Title VII) of the 1964 Civil
Rights Act: whether the Act shall remain a potent tool
for equalization of employment opportunity or shall be
reduced to mellifluous but hollow rhetoric.
34
discriminated against are essentially the same and deserve the
same degree of protection whether employment opportunities are
denied explicitly and intentionally or inadvertently. it is
for such reasons that this and other courts have made it clear that
where there is evidence of a racial impact or classification, no
30
evidence of an overt or covert intent to discriminate is necessary — ' * S
29/ (Continued)
The pattern of racial discrimination in employment
parallels that which we have witnessed in other areas.
Overt bias, when prohibited, has ofttimes been supplanted
by more cunning devices designed to impart the appearance
of neutrality, but to operate with the same invidious
effect as before. . . . (420 F.2d at 1237-38)
See also the Supreme Court’s opinion, 401 U.S. at 431.
S®e ' 2̂3.*' Norwalk CORE v. Norwalk Redevelopment Authority.
395 F.2d 920, 931 (2d Cir. 1968) (finding irrelevant the fact that
discriminatory effect of an urban renewal housing plan is "accidental "
since Equal Protection Clause prohibits thoughtlessness and not -just
governmental design to discriminate, citing Hobson v. Hansen, infra)-
T~,n.nedT^^vrk Homgs Ass’n- v. Lackawanna, 436 F.2d 108, 114 (2d Cir 1970) (holding with regard to the City's provision of facilities
that even if the discriminatory effect resulted from "thoughtlessness
rather than a purposeful scheme, the City may not escape responsibility
for placing its black citizens under a severe disadvantage which it
cannot justify"; Southern Alameda Spanish Speaking Org. v. Union
Ci^, 424 F . 2d 291, 295 (9th Cir. 1970) (if result of zoning bv
referendum is discriminatory a substantial Constitutional question
^presented without regard to racial motive); Hobson v. Hansen.
Ini p •,?"??: j01' 497‘ <D -D -C - 1967). aff'd sub nom Snmck v. Hobson.408 F.2d 175 (D.c. Cir. 1969) {". . . the arbitrary quality of
thoughtlessness can be as disastrous and unfair to private rights and
the public interest as the perversity of a willful scheme"; Hawkins
v^ Town of Shaw, 437 F.2d 1286, 1291-92 (5th Cir. 1971) (pendTHS----
decision after rehearing en banc) (finding a prima facie case of
discrimination made out on the basis of statistical disparities in
the provision of municipal services to different racial groups although
the record showed no "bad faith, ill will or . . . evil motive")-
Cgrmical v. Cravep,9th Cir. No. 26, 236, Nov. 4, 1971 (holding that
where test disqualified disproportionate number of veniremen
from jury service, the fact that the system was not designed with
discriminatory intent was irrelevant); Jackson v. Godwin, 400 F.2d
529 (5th Cir. 1968). Cf. Powell v. Power. 436 F.2d 84, 88 n. 7
(2d Cir. 1970). See also, Gaston County v. United States, 395 U.S. 285
(1969); Smith v. Texas, 311 U.S. 128, 132 (1940) (racial discrimination
in jury selection unconstitutional "whether accomplished ingeniously or ingenuously").
35
And virtual^ every decided case involving the use of tests
by public employers has held that where a test has a substantially
different impact on different racial groups and cannot be justi
fied as job-related, its use violates the Equal Protection Clause,
whether or not an intent to discriminate can be found. ^
31/
i .Thus m Carter v. Gallagher, 3 CCH E.P.D. f8335 (8th Cir.
' M f>™ing m pertinent part, 3 CCH E.P.D. f8205 (D. Minn.
1971), panel opinion adopted in pertinent part en banc, No. 71-1181
8th Cir. Jan. 7, 1972, slip op. p. 11, involving fire depart
ment hiring practices, the court noted that there had been "no
express finding of bad faith or evil motives," and held that
under 42 U.S.C. §§1981 and 1983 there was no "requirement that
discrimination be willful or intentional" (p. 6993-203) in
Penn v. Stumpf, 303 F. Supp. 1238, 1243-44 (N.D. Cal. 1970), the
court found that while there was "no question here of overt
racial classification," a selection procedure "may appear on
its face to be fair and neutral, but if in its application a
discriminatory result ensues, the procedure may be constitutionally
impermissible" (police dept, tests). Arrington v. Mass. Bav
iransp. Authority, 306 F. Supp. 1355, 1358 (D. Mass 1969),
held irrelevant the fact that there was "no contention of any
intent to discriminate against black or Spanish-speaking persons
by means of this test," on the grounds that:
. . . It is not enough that the factors producing the
classification and the consequent inequality are them
selves objectively neutral and without a background of
even latent discriminatory purpose: when the effect
is to deprive some citizens of rights that should be
equally available to all, then there must be a compel
ling justification. . . . "
In Western Addition Community Organization v. Alioto, 330 F.
Supp. 536, 539-40 and n. 4 (N.D. Cal. 1971), the court held'that:
. . . where the hiring practice of a public agency (even
though it does not intend to discriminate against
minority groups) has the effect of producing a de facto
pattern of racial discrimination, such a discriminatory
effect . . . renders the method of selection sufficiently
suspect to make a prima facie case of unconstitutionality."
(330 F. Supp. at 539)
Ca-stro v. Beecher. 4 CCH E.P.D. f~569, clarified at f7570, judgment
modified at [̂7589 (D. Mass. 1971), struck down police department
employment tests on the ground that they had the consequence of
disqualifying minority applicants at a substantially higher rate
than wnite applicants and were not job-related, finding specific-
that 'plaintiffs need neither allege nor prove that defendants
36
Supreme Court has made it similarly clear that under Title VII of
the 1964 Civil Rights Act (42 U.S.C. § 2000-e), which outlawed
discrimination by private employers,
"good intent or absence of discriminatory intent does not
redeem employment procedures or testing mechanisms that
operate as 'built-in headwinds' for minority groups and
are unrelated to measuring job capability."
32/Griggs v. Duke Power Co., 401 U.S. 424, 432 (1971).
The Griggs decision, as well as other testing cases decided
under Title VII, is directly applicable here, since the prohibitions
^9 îri£3t employment discrimination embodied in the Act, governing
private employers, coincide with those embodied in the Fourteenth
Amendment, governing public employers. It would be anomalous in
deed if public employers had a lesser obligation than private to
32a/afford fair treatment to all ethnic groups.
31/ (Cont'd)
acted with intent or motive to discriminate," citing Griqgs. infra
(4 CCH E.P.D. f7569 at p. 5276 n. 4; pp. 5282-83). Both Armstead
v. Starkville Mun. Sep. School Dist., 325 F. Supp. 560, 570 (N.D.
Miss. 1971), and Baker v. Columbus Mun. Sep. School Dist.. 329
F^ Supp. 706, 720—21 (N.D. Miss. 1971) held, wholly independent
of any findings of intentional discrimination, that use of the
Graduate Records Examination (Armstead) and N.T.E. (Baker) to select
teachers violated Equal Protection because they disqualified blacks
at disproportionate rates and were not job-related.
32/ See also, e.g., Johnson v. Pike Corp., 4 CCH E.P.D. f7517 at pp.
5048, 5050 (Cent. D. Cal. 1971), noting that Griggs was not limited
to situation of previous overt discrimination and holding unlawful
a policy of firing persons whose wages were garnisheed even though
the discrimination effected by such policy was "not really intended
by the employer;" Robinson v. Lorillard Corp., 444 F.2d 791, 796-97
(4th cir. 1971); Hicks v. Crown Zellerbach Corp., 319 F. Supp 314
320 (E.D. La. 1970^ " ~ ~ ---
3_2a/ Thus Fourteenth Amendment prohibitions against discrimination
kave keen found "at least as great as those levied upon private
employers in Section 703" of the 1964 Civil Rights Act, Baker,
s.yj?rg r*- 31. citing and relying on Griggs and other Title VII cases,
329 F. Supp. at 721. The Act is if anything narrower in its
coverage since it expressly permits the use of tests provided they
are not "designed, intended or used to discriminate...." 42 U S C § 2000e-2 (h) .
37
B. Where Employment Tests are Shown to have a Significant and
Substantial Discriminatory Impact on Ethnic Minority Groups
They Violate the Equal Protection Clause if They Cannot be
Shown to be Job-Related.
(1) The Court applied the proper standard in determining that
an adequate case of discriminatory impact had been made out.
Defendants contend that the court below applied a standard of
statistical significance, demanding perfect equality in treatment
of the races, and that this standard is inconsistent with the law.
(Brief pp. 21-26). As noted supra pp. 26-28, the court did not
in fact apply such a standard but relied instead on its findings of
substantial" and "gross" disparities based on the survey results
and on the multiplier effect of the requirement that candidates pass
through a succession of tests. This standard was completely
consistent with decided cases in the employment discrimination area,
which have applied the principle that where an employment policy has
a differential impact on different racial groups; and cannot be
justified on non-racial grounds, a case of discrimination is made out.
The instant case is unique not because a particularly strict
standard was applied but, rather, because the discriminatory impact
of defendants' examinations was so thoroughly documented. Most
employment discrimination cases, including the Supreme Court's decision
in Griggs v. Duke Power, supra, have struck down tests without any
comparable showing of discriminatory impact. Statistics on comparative
pass rates of different ethnic groups have usually been either wholly
unavailable or else hopelessly inadequate as the basis for drawing
valid conclusions as to the overall impact of the tests. The courts
have therefore tended to rely on assumptions (based on such factors
as the nature of the tests, the different cultural and educational
backgrounds of different racial groups, and low percentages of
minority persons in the work force) that there is some significant
38
discrepancy between pass rates of different groups. Thus in
Griggs the Supreme Court struck down tests which the record re
veals only three candidates took -- two black and one white --
33 /all of whom failed.— The Court simply had to assume that
the tests operated "to disqualify Negroes at a substantially
higher rate than white applicants" 401 U.S. at 426 (emphasis
added), on the basis of findings as to how blacks and Puerto
Ricans had fared on such tests elsewhere (id. at 430, n. 6).
The Court concluded that they were discriminatory since
"[b]asic intelligence must have the means of articulation to
manifest itself fairly in a testing process" (id. at 430). In
Penn v. Stumpf, supra n. 31 , police promotional tests were
found discriminatory on the basis of disparities between
minority representation in the Oakland City population and
on the Police Department, and an allegation that significant
numbers of blacks had applied. In Carter v. Gallagher, supra
in an opinion subsequently adopted en banc, n. 31 , the Eighth Circuit found/Fire Department examinations
discriminatory on the basis of a showing that the Department
was all-white, whereas about 6% of Minneapolis' population
was non-white, and that the test used a formal English vocabu
lary and would thus tend to eliminate blacks at a higher rate
see Brief for Petitioners, p. 6 and n. 3, and see record
filed in the U. S. Supreme Court.
39
than whites.34y In Castro v. Beecher, supra n. 31 , the court
concluded that police examinations were discriminatory against
all "minorities which did not share the prevailing white
culture, simply because of their verbal, academic and multiple
choice nature.— / jn Western Addition Community Organization v
Alioto, supra n. 31, the court relied on disparities between
minority representation on the Fire Department and in the
area's general population, ignoring statistics which showed
that of the two ethnic groups allegedly discriminated against,
one, the Mexican-American, had passed at virtually the same
rate as the white group (34% as compared to 36%). * 3 4
34y
3 CCH E.P.D. f8338 at 6993-203; 3 CCH E.P.D. *8205 at
p. 6669. While some comparative pass rate statistics were
available, the district court apparently felt it could not
rely solely on these because the sample of identifiable
minority candidates was so small. 3 CCH E.P.D. f8205 at
p. 6669.
35/ 4 CCH E.P.D. ^[5283; see also p. 5284. The court noted
in addition that black and Spanish-surnamed candidates had
done less well on the examinations than white candidates, but
this was not central to its finding of discrimination. 4 CCH
E. P.D. f7659 at p. 5282.
330 F. Supp. at 538. In Gregory v, Litton Systems. 316
F. Supp. 401, 463 (C.D. Cal. 1970), the court struck down an
employer's policy of disqualifying applicants who had been
arrested a number of times without any evidence as to the
impact of this policy on actual applicants. The court simply
assumed that the policy denied black applicants "equal oppor
tunity for employment," and disqualified a "substantial and
disproportionately large number of Negroes," on the basis of
national arrest figures showing that Negroes had a higher
arrest rate than whites. In Johnson v. Pike Corp., 4 CCH
E.P.D. f7517 at p. 5049 (C.D. Cal. 1971), the court found a
violation of Title VII in a rule authorizing discharge of
employees whose wages had been garnisheed on the grounds that
"available information" indicated that "the proportion of
racial minorities among the group of people who have had their
wages garnisheed is significantly higher than the proportion of racial minorities in the general population."
40
The few employment cases which discuss statistical dis
parities between the pass rates of different racial groups
support a finding of discriminatory impact in the instant
c a s e . ^ And cases in such areas as jury discrimination
and reapportionment 39/ have established legal standards
which compel a finding that the statistical disparities shown
here make out a prima facie case of discrimination.
See, e.g. , Carter v. Gallagher, supra nn. 31,34, where the district
court found minority groups apparently did "substantially less
well" on the basis of partial statistics indicating that 27%
of identified minority group members passed as against an
overall pass rate varying between 40% and 65%. 3 CCH E.P.D.
f8205 at 6669.
See, e.g., Turner v. Fouche, 396 U.S. 346, 360 (1970);
Whitus v. Georgia, 385 U.S. 545, 552, n. 2 (1967); Jones v.
Georgia, 389 U.S. 24 (1967); Sims v. Georgia, 389 U.S. 404,
407 (1967). In U.S. v. Criminal Courts of New York, 442
F.2d 611 (2d Cir. 1971), in refusing to overturn criminal
contempt convictions on the grounds that the grand jury
before which the contempts had occurred was unrepresentative,
this court made it clear that it was dealing with the right
of the defendants to a fair tribunal and not the rights of
minority group members to serve as jurors. The primary issue
was, therefore, whether a case of bias had been made out and
the court held it had not since there was no apparent ground
for the "assumption that grand jurors predominantly of one
race or those over 35, cannot provide a fair hearing for per
sons of another race or generation" (442 F.2d at 616). And
in considering New York's system of jury selection the court
again noted that "the only plausible basis for Chestnut's
equal protection claim is the possibility that a grand jury
deficient in members of his race would deal with him dif
ferently. . . . " (442 F.2d at 618, n. 12) It relied also on
the fact that there was no denial "that the racial imbalance
of the [jury] list was not the result of 'an exclusionary
device . . .' but was due to New York County's system of
voluntary grand jury service" (442 F.2d at 617-18), a factor
distinguishing it from the jury cases cited supra. Indeed
the court specifically found that the under-representation
of blacks "resulted solely from their relative lower propen
sity to volunteer" (442 F.2d at 618, n. 12).
-23*1 See, e.g. , Kirkpatrick v. Preisler, 394 U.S. 526, 530-31
(1969); Wells v. Rockefeller, 394 U.S. 542 (1969).
41
The court's standard is not only consistent with decided
authority, but seems the only reasonable standard in the em
ployment testing area. In recent years a number of studies
have documented the fact that most tests discriminate unfairly
against blacks and Puerto Ricans because they are designed
for persons with different cultural and educational back-
40/grounds. Recent studies have also documented the fact
that there is generally very little relation between employ
ment tests and 30b performance,^-^ particularly when such
42 /tests are given to mixed racial groups.— ' And finally.
40/ See, e.g., Kirkpatrick, et al.. Testing and Fair Employ
ment 5 (1968); J. Coleman, Equality and Educational Opportunity
219-20 (1966); authorities collected in Cooper and Sobol,
Seniority and Testing under Fair Employment Laws, 82 HARV. L.
REV. 1598, 1639-41 pp. 11,13,14,15,16, 17 (1969); Hobson v.
Hansen, 269 F. Supp. 401, 484-85 (D.D.C. 1967), aff'd sub nom.
Smuck v. Hobson. 408 F.2d 175 (D.C. Cir. 1969). See also
Barrett affidavit, App. 36-38a.
41/ See, e.g., E. Ghiselli, The validity of Occupational Apti-
titude Tests 51, 57 (1966). In Griggs the Supreme Court notedthat:
"The facts of this case demonstrate the inadequacy
of broad and general testing devices as well as the
infirmity of using diplomas or degrees as fixed
measures of capability. History is filled with examples
of men and women who rendered highly effective perform
ance without the conventional badges of accomplishment
in terms of certificates, diplomas, or degrees."
(401 U.S. at 433)
42/ See generally authorities cited in Cooper and Sobol,
supra n. 40at 1643-46 (1969); Hobson v. Hansen, supra n. 40,
269 F. Supp. at 484-85.
42
workable standards have been developed by administrative
agencies charged with enforcing laws against discrimination,
for ensuring that tests are in fact job-related and non-
discriminatory. f Given this history, it does not seem
unreasonable to demand that if examinations are shown to have
a substantial discriminatory impact on ethnic minority groups,
some determination be made as to whether they can be justified
as anything other than arbitrary selection devices. This seems
particularly appropriate in light of (1) the importance of
employment to the individual; (2) the fact that the State's
essential interest is in obtaining qualified employees, a goal
43/
See, e.g., the Guidelines on Employee Selection Procedures
issued by the United States Equal Employment Opportunity Com
mission, the agency charged with enforcement of Title VII
of the 1964 Civil Rights Act, 35 Fed. Reg. 12333 (Aug. 1,
1970) [hereinafter cited as the E.E.O.C. Guidelines). Identi
cal Guidelines have been adopted by the Office of Federal
Contract Compliance, charged with enforcement of Executive
Order 11246 against discrimination by government contractors,
35 Fed. Reg. 19307 (Oct. 2, 1971), as well as by state anti-
discrimination agencies (see, e. q. , Pennsylvania's Guidelines
at 1 Pa. Bull. 2005, Oct. 16, 1971, CCH E.P. Guide *5194;
California Fair Employment Practices in CCH E.P. Guide
f20, 861; Colorado Civil Rights Commission Policy Statement
on the Use of Psychological Tests in CCH E. P. Guide f21,060.
43
furthered by a demand for job-related tests; and (3) this Nation's
traditional insistence that employment selection and promotion
44/
decisions be made on the basis of merit.
(2) The Court Properly Determined That Examinations Which Have
Such a Discriminatory Impact Violate the Equal Protection
Clause if They Cannot Be Shown to Be Job-Related.
The Examiners claim that the court erroneously ruled that
once their examinations had been shown to have a substantial
discriminatory impact on blacks and Puerto Ricans the burden
shifted to defendants to show a "compelling state interest" in
maintaining such a system, arguing that the court should instead
have applied a "rational relationship" test (Brief pp. 36-41).
44/cndeed, even if defendants' examinations had not discriminated
substantially against blacks and Puerto Ricans, the court's find
ing that they could not be shown to be job-related established a
violation of Due Process and Equal Protection. Schware v. Board
of Bar Examiners, 353 U.S. 232 (1957); Chaney v. State Bar of
California, 386 F . 2d 962 (9th Cir. 1967) (dictum) . Thus in Armstead,
supra, the court specifically held that defendants' uses of the
GRE violated the Due Process Clause because they constituted
"arbitrary and unreasonable qualifications for employment" wholly
"apart from [their] discriminatory aspects" (325 F. Supp. at 570).
Similarly in Baker, supra, the court concluded that "apart from
its discriminatory aspects," use of the N.T.E. without a demon
strated correlation between test scores and job performance violated
the Due Process Clause. 329 F. Supp. at 722.
Defendants' examination system was also shown to be in viola
tion of state law, wholly apart from any findings of discriminatory
impact. The State Constitution provides that civil service appoint
ments and promotions "be made according to merit and fitness. ..."
(N. Y. State Const. Art. V §6), and N. Y. State Educ. Law §2569(1)
(1967) provides that the Board of Examiners "shall periodically
review the validity and reliability of examinations as well as
44
However, while plaintiffs maintain that the compelling state
interest test is appropriate, this court need not reach that
issue. The court below rested its decision on findings that the
Examiners could demonstrate no relationship between their tests
and the purpose for which they were used, and that indeed the
Examiners had failed to create job-related examirn tions. These
findings satisfy the rational relationship test. Thus the court
found that the Examiners had failed to make out a claim for con
tent validity, and that available evidence (including the pilot
predictive validity study) indicated that their examinations were
not related to job performance. The court therefore did not have
to reach the issue as to whether the Examiners had the burden of
providing predictive validity data demonstrating a correlation
between test scores and job performance. Such data would obviously
45/
have been required under the compelling state interest test.
Application of that test would also have required a showing that
there was no alternative examination system which would satisfy
44/ (Cont'd)
examination procedures. ..." Since the court found defendants'
examinations were not job-related and had never been validated,
they were in clear violation of the State law and Constitution.
These State law claims were presented below, based on the court's
pendent jurisdiction, but were not decided (App. 140a, 151-53a, 182a).
45/ See, e.g., Arrington, Baker, Robinson v. Lorillard. United
States v - Jacksonville Terminal Co., all discussed infra pp. 49-51.
45
the state's interest in selecting the best-qualified applicants
without disqualifying disproportionate numbers of blacks and Puerto
46/
Ricans. Again the court below did not find it necessary to
reach this issue. Thus, even if the rational relationship test
urged by the Examiners is applied, the decision below must be
£2/upheld.
In any event, the compelling state interest test is clearly
appropriate in the circumstances of this case. It is established
law that wherever different racial groups are differently treated,
48
the burden of justification is placed on the responsible officials,
46/ Under the compelling necessity test a classification must be
found necessary to accommodate the legislative purpose. See, e.g.,
Kramer v. Union Free School District. 395 U.S. 621, 663 (1969).
An absence of alternative devices to achieve the same end must be
found. See McLaughlin v. Florida, 379 U.S. 184 (1964). For employ
ment discrimination cases applying this test see, e.g., Pickens.
United States v. Bethlehem Steel Co., Robinson v. Lorillard. all
discussed infra pp. 49 - 5 1 .
4.7 / Compare Western Addition v. Alioto. supra, n. 31 p. 36 , where
the court found unintentionally discriminatory tests placed the
burden on the employer of "showing a reasonably necessary connec
tion between the qualities tested in the ... examination and the
actual requirements of the job to be performed" (330 F. Supp. 539-40);
Qastro v. Beecher, 4 CCH E.P.D. 17569 at p. 5282, supra n. 31 p. 36
where the court struck down police examinations on the grounds that
they had not been prepared by persons with special training in test
design, did not appear to be content valid, and had not been
empirically validated.
48_/ This principle was first established in the jury discrimination
cases. See, e^. , Norris v, Alabama. 294 U.S. 587 (1935), and cases
cited supra n. 38 . In these cases courts have placed on the
government the burden of justifying significant under-representation
46
and is satisfied only by a showing of a compelling necessity.
Such a test is particularly appropriate where, as here, fundamental
50/
interests of the minority group are at issue.
The Examiners claim that the compelling state interest test
is appropriate only where there is an explicit statutory racial
classification, but it is clear that it has not been so limited.
49/
48/ (Cont'd)
of minority groups on juries and have refused to make assumptions
that blacks are less likely to meet the qualifications (which
typically require, for example, that the juror be honest and
intelligent, of good character, literate, and never convicted of
an offense of moral turpitude). See, e .g., Patton v. Mississippi,
332 U.S. 463, 468 (1947); Brown v. Allen. 344 U.S. 443, 471 (1953);
Jones v. Georgia, 389 U.S. 24 (1967).
Similarly, in a testing case, courts cannot assume that blacks
do less well because they are less qualified but must insist that
the employer explain and justify the differential passing rates of
different racial groups. Compare the Supreme Court's conclusion
in Griggs that tests on which blacks did badly were discriminatory
since "[b]asic intelligence must have the means of articulation to
manifest itself fairly in a testing process" (401 U.S. at 430).
49/ See, e .g ., McLaughlin v. Florida, 379 U.S. 184, 192 (1964);
Korematsu v. United States, 323 U.S. 214, 216 (1944); Bolling v .
Sharpe, 347 U.S. 497, 499 (1954) (all noting that racial classifi
cations are 'constitutionally suspect' and subject to the 'most
rigid scrutiny'); Loving v. Virginia, 388 U.S. 1, 9 (1967) ("very
heavy burden of justification"); Graham v. Richardson, 403 U.S. 365
(1971); Kennedy Park Homes Ass'n v. City of Lackawana, 436 F.2d
108 at 114 (2d Cir. 1970).
50/ Compare Harper v. Virginia Board of Elections, 383 U.S. 663
(1966); Shapiro v. Thompson, 394 U.S. 618 (1969).
47
It has been found applicable wherever a racial classification
results however neutral the statute may be on its face and how-
51/
ever benign the administrators' intent.
And in the employment discrimination area specifically, the
courts have applied this standard in reviewing employers' use of
selection devices shown to have a discriminatory impact, without
regard to the employers' intent. Thus in Carter v. Gallagher.
suPra n. 3i , the court quoted and applied the "business necessity"
standard laid down in Griggs (3 CCH E.P.D. *8335 at p. 6993-205),
.51/ In Dandridge v. Williams. 397 U.S. 471, 485, n. 17 (1970),
the Supreme Court specifically noted that a welfare statute
"infected with a racially discriminatory purpose or effect" would
be "inherently suspect." In Goodwin v. Wyman (3-judge court),
330 F . Supp. 1038, 1040 (S.D. N.Y. 1971), the court found that
under Dandridge, a welfare regulation with a racially discrimi
natory effect was “inherently suspect" and required a "compelling
state interest" for justification. In Hawkins v. Town of Shaw.
supra, n. 30 , the Fifth Circuit held that a "compelling state
interest" test was required where the results of Shaw's administra
tion of municipal services were discriminatory even though no
discriminatory intent was shown, 437 F.2d at 1288, 1292-93. Hobson
v. Hansen, supra n. 30 , held that "thoughtless" discrimina
tion placed the burden on defendants of producing "convincing
justification" 269 F. Supp. at 497, 498. See also Jackson v .
Godwin, supra n. 30 , 400 F.2d at 537; Kennedy Park Homes Ass'n v
City of Lackawana. 436 F.2d 108 at 114 (2d''~Cir. 1970) . -----------
The Examiners rely on Johnson v. New York State Education
Department, 449 F.2d 871 (2d Cir. 1971), but in Johnson, as in
Dandridge, supra, there was no racial issue. The only classifi
cation involved was drawn on the basis of the grade level of
children. While claims of discrimination against the indigent
were made, the court found that the real issue was not one of
discrimination but, rather, of whether the Equal Protection Clause
guaranteed that indigent children must receive free textbooks.
48
and upheld various aspects of the district court's decision based
on the compelling state interest test (striking down, e.g.,
certain arrest record and education eligibility requirements, 3
CCH E.P.D. 18205 at p. 6681). In Arrington, supra, n. 31,
the court held that tests resulting in unintended discrimination
were "constitutionally suspect," requiring for justification a
"demonstrated correlation between scores ... and ability to per-
52/
form" (306 F. Supp. at 1358).
A similar strict test has been applied in Title VII cases.
In Griggs v. Duke Power, supra, the Supreme Court held that:
The touchstone is business necessity. If an
employment practice which operates to exclude
Negroes cannot be shown to be related to job
performance, the practice is prohibited.
(401 U.S. at 431)
Cases since Griggs have made it clear that for an employer to
establish a claim of "business necessity" he must make a very
/// in Baker, supra n. 31 , the court struck down use of the NTE
because defendants had not satisfied their "very heavy burden of
justification, which included the burden of showing a "manifest
relationship" between scores and job performance (329 F. Supp. at
721). In Armstead, supra n. 31 , the court held that the GRE's
discriminatory impact on teacher applicants placed a "very heavy
burden of justification" on the employer to show an "overriding
purpose." And in Pickens v. Okolona Mun. Sep. School Dist.. M.D.
Miss. E. Div. No. EC6956-K (Aug. 11, 1971), the court held that
use of the NTE where it had a discriminatory impact placed the
burden on the employer to show an "absolute, overwhelming neces
sity* ••• would have to be shown that [it was] the only way
it was possible to obtain competent and worthy teachers." (Slip
op. pp. 17-18)
49
strong showing both that test scores are correlated with job
performance and that there is no less discriminatory alternative
available. Thus United States v. Jacksonville Terminal Co.. 3
E.P.D. f8324 (5th Cir. 1971), struck down tests on the ground
that the employer's validity study was inadequate, holding that
— demanded "more substantial proof, most often positive
empirical evidence, of the relationship between test scores and
job performance (p. 6993-149). And in United States v .
Bethlehem Steel Corp., 446 F.2d 652 (2d Cir. 1971), this court
defined business necessity as follows:
Necessity connotes an irresistible demand. To be
preserved, the seniority and transfer system must
not only directly foster safety and efficiency of
a plant, but also be essential to those goals.
If the legitimate ends of safety and efficiency
can be served by a reasonably available alterna
tive system with less discriminatory effects, then
the present policies may not be continued.
(446 F.2d at 662)53/
53/ Robinson v. Lonllard C o m .. 444 F.2d 791 (4th Cir. 1971),
defined Griggs' business necessity test as follows:
... The test is whether there exists an overriding
legitimate business purpose such that the practice
is necessary to the safe and efficient operation
of the business. Thus, the business purpose must
be sufficiently compelling to override any racial
impact; the challenged practice must effectively
carry out the business purpose it is alleged to
serve; and there must be available no acceptable
alternative policies or practices which would better
accomplish the business purpose advanced, or accom
plish it equally well with a lesser differential
racial impact.
50
Since the Examiners failed to demonstrate a correlation
between test scores and job performance, and to show that no
job-related system could be developed which would be less dis-
54/
criminatory in impact, their examination system cannot be
upheld under the compelling state interest test. However, as
noted supra, the court need not reach this issue since the find
ings below satisfied the rational relationship test.
53/ (Cont'd)
★ ★ ★
... It should go without saying that a practice is
hardly "necessary" if an alternative practice better
effectuates the intended purpose or is equally
effective but less discriminatory. Thus, with regard
to testing, the Equal Employment Opportunity Commis
sion guidelines stipulate that:
Where technically feasible, a test should be
validated for each minority group with which
it is used; that is, any differential rejection
rates that may exist, based on a test, must be
relevant to performance on the jobs in question.
29 C.F.R. §1607.4(a) (444 F.2d at 798 and n. 7)
Lorillard further notes that:
... although there undoubtedly are significant
costs involved in validating tests Griggs requires
that employment tests be abandoned if not specif
ically validated as job-related.
(444 F .2d at 799 n. 8 )
54/ The E.T.S. proposal, supra p. 20 , provides a strong indica
tion that an alternative examination system could be developed
which would be job-related but would not exclude ethnic minorities
at disproportionate rates (infra, p. 16a).
51
POINT III
THE COURT'S FINDING THAT DEFENDANTS' EXAMINATIONS
HAD NOT BEEN SHOWN TO BE AND APPARENTLY WERE NOT
JOB-RELATED WAS CLEARLY SUPPORTED BY THE RECORD
The Examiners' argument that the court erred in its
findings as to job-relatedness is based on a wholly distorted
version of the evidence and of the basis for the court's
findings (Brief pp. 26-36). While we argue below that the
court's findings were clearly correct, there can be no question
that they were supported by extensive evidence, briefly sum
marized at 4-5,8,10-11,supra, and therefore must be upheld as
not clearly erroneous.
In the proceedings below the Examiners rested their claim
for job-relatedness primarily on: (1) a number of research
reports and studies which, it alleged, demonstrated the
validity of their examinations, and provided empirical data
showing that test scores were related to job performance
(see p . 7, supra) and (2) on brief affidavits submitted by
four experts they had consulted, who based their opinions on
documents submitted to them by the Examiners (see p.7,
supra). While relying on the research reports, the Examiners
refused to produce them until ordered to do so by the court
pursuant to plaintiff's discovery request. It is only now
that plaintiffs have had a chance to subject the reports to
52
expert analysis, and the court below has rejected them as
entirely w o r t h l e s s t h a t the Examiners argue that these
reports were not central to their claims for validity (Brief
30-31). Analysis of the Board's research efforts was clearly
relevant not only to the issues of content and predictive
validity but also to the credibility of its other claims
with respect to the efforts it made to establish content
validity.
Plaintiffs' evidence on the issue of ]ob-relatedness
included numerous affidavits by experts who had analyzed,
inter alia, all documents submitted by the Examiners in sup
port of their claim of job-relatedness, all their alleged
research reports, and all the documents on which the Examiners'
experts had based their opinions?-^-/ The court's decision
was therefore based not simply on its own lay analysis of
15/ The court found that none of the reports and studies
even related to validity, content or predictive, with one
exception - a pilot predictive study which demonstrated no
correlation between test scores and job performance (App.
215-16a).
See App. 35-39a, 107-20a, 155-64a. What the Board terms
the court's "superficial critique" (Brief p. 30) of its re
search reports was based on an extensive analysis of these
reports by one of plaintiffs' experts. See App. 158-62a,
App. Ex. 15.
53
reports and examinations, but on consideration of expert
evidence offered by both sides, as well as extensive brief
ings on the professional testing literature.-^7
It is clear from the evidence below and the testing
literature that there are two accepted ways of showing job
relatedness or validity. One is content validation, which
depends on demonstrating that the test tasks are comparable
to the tasks of the job being tested for, so that one can
assume that test performance will be predictive of job per
formance. The other is predictive validation (sometimes
called "criterion-related" or "empirical" validation), which
depends on demonstrating an actual correlation between test
scores and job performance.
The Examiners now rely solely on a claim of content
validity (Brief pp. 26-27). But such a claim can be sustained
only by a showing that adequate procedures have been followed
in developing job analyses, test specifications, and, finally,
related test questions.-^7 Here the court found that the
— / See, e.g., Plaintiffs' Memorandum in Reply to Defendants'
Memorandum Opposing Motion for Preliminary Injunction, Ind.
Doc. No. 51, filed November 6, 1970.
58/
See generally R. THORNDIKE, Personnel Selection &
Measurement Techniques (1949) at 14, 15, 50; CRONBACH, Essentials
of Psychological Testing (3d Ed. 1970) at 115-50.
54
Examiners had not in fact followed such procedures on the
basis of a record which included evidence demonstrating: (1)
that defendants had never performed any adequate job analysis
for any position,^/ or made an adequate determination of the
criteria being tested for.-^/ t'>\ .y (2) that tests were designed and
administered by an untrained staff, composed of teachers and
supervisors selected on an ad hoc basis, who were subject to
the supervision of a Board which had only one or at times two
persons on its research staff; ^ and (3) that while some
examination questions seemed superficially related to the
jobs at issue, many could in no sense be considered typical
of or related to 3ob t a s k s . ^ Finally, it is most significant
59/ See pp.4,8,10-1], supra.
|0/ The court cited the Strauss affidavit, referred to in the
Examiners' Brief at pp. 28-29, merely to rebut the Board's
claim that the meeting he attended served the purpose of
helping develop test criteria. See App. 214a, 103-05a.
§1/ See pp. 4 8 supra,
62/
See A££. 217-18a, 36-37a, 114a. while the
court based its conclusions primarily on the inadequacy of the
Examiners' procedures, it also considered the examinations
themselves, including the Nov. 1970 Elementary School Principals'
Examination, which were submitted to it, and subjected to
expert analysis and extensive briefing.
55
that the Examiners' own experts "did not conduct the kind of
analyses which could support a claim of content validity"
f) 3 /(App. 157a ') and, indeed, did not even claim content validity
(p. 7 , supra).
Moreover, although the court below did not rely on the
Examiners' conceded inability to provide any predictive vali
dation data for its examinations, this in itself required a
finding that the examinations could not be shown to be job-
related. Expert testimony below, supported by recognized
authorities in the testing area, indicated that where tests
are used as devices to predict performance, rather than simply
64/to assess proficiency, predictive validation is essential.—
— As established by the record below and supported by the
literature, a claim for content validity cannot be supported
by a mere comparison of an examination with a list of job
duties (See App. 162-63a, 157-58a, 112-14a). A. ANASTASI,
Principles of Psychological Testing 100-101, 104 (3d Ed. 1968).
— ^ See p. 8 , supra.
The court noted in its opinion, App. 206a, the published
views of Dr. Thorndike, one of the Examiners' experts below,
to the effect that:
"Whenever a test is being tried for selection of
personnel for some job specialty, it is most desirable
that it be validated empirically. Experimental evidence
is called for to show that the test is in fact effective
in discriminating between those who are and those who
are not successful in a particular job." R. L. THORNDIKE,
Personnel Selection 5-6 (1949) (emphasis added).
See also R. L. THORNDIKE and E. HAGEN, Measurement and
Evaluation in Psychology and Education 616-641 (1969);
O. G. STAHL, Public Personnel Administration 11 (5th ed. 1962)
These standards, accepted in the testing profession, have been
incorporated by administrative agencies and courts in assessing
56
The need for predictive validation cannot be avoided simply
by calling a test an achievement or proficiency test rather
than an aptitude test, as the Examiners suggest. The Board
uses its tests in an attempt to select the ablest future
administrators. This is clearly a predictive purpose, and
it should be supported with predictive validity data.^'
Content validation is sufficient only where the test
tasks can truly be said to be comparable to 30b tasks (and
even then predictive validation is considered preferable).
64 / (Cont'd)
the fairness of test use in employee selection. Thus the EEOC
Guidelines on Employee Selection Procedures. 35 Fed. R e a 12333
at § 1607.4(c) (Aug. 1, 1970), supra n. 43, demand that employers have available:
empirical data demonstrating that the test is predictive
of or significantly correlated with important elements
of work behavior which comprise or are relevant to the
job or jobs for which candidates are being evaluated.
As noted supra n. 43, identical standards have been adopted
by the OFCC and by state anti-discrimination agencies. Cases
incorporating or approving the EEOC Guidelines include, in the
public employment area, Carter, Baker, and Armstead, supra n. 31 .
Arrington_v. Massachusetts Bay Transportation Authority,
suPra n.31, adopts a similar standard. The Supreme Court
approved the guidelines in Griggs, supra, as have other courts
in Title VII cases, e.g., united States v. Jacksonville Termi-
nal Co., supra p .50 ; Hicks v. Crown Zellerbach" supra n~. 32.
See A. ANASTASI, Psychological Testing 120-121 (3d ed. 1968)
57
A typical example of reliance on content validity is use of
a typing test to select typists: since the test task is
essentially the same as the job task, the assumption that it
is predictive of job performance may be warranted. Defendants'
tests, by contrast, cannot be considered adequate samples of
the jobs to be performed. As indicated in the court’s
opinion, many of the questions are not even superficially
related to the jobs at issue (App. 217-18a, 232-33a, n. 23).
And even those questions which are superficially related
cannot be considered equivalent to the tasks the person
would actually have to perform on the job.
Moreover, the court's finding that the Examiners' only
attempt at predictive validation "showed that there was little
or no correlation between success on the tests and job success,"
(App. 216a, see also App. 159-61a), indicates that the Board
not only failed to show job-relatedness, but that the tests
were in fact not job related.
58
POINT IV
THE GRANT OF PRELIMINARY RELIEF, BASED
UPON THE COURT'S FINDINGS WITH REGARD
TO IRREPARABLE INJURY AND THE LIKELI
HOOD OF SUCCESS ON THE MERITS, MUST BE
UPHELD AS A PROPER EXERCISE OF DIS
CRETION
The trial court's decision to grant preliminary relief
can be reversed only if it constituted an abuse of discretion.
And appellate courts should be reluctant to reverse in such
cases because of the
"flexible interplay between the likelihood of
irreparable harm to the movant and the court's
belief that there is a 'reasonable certainty1
that the movant will succeed on the merits at
a final hearing. . . . "
Packard Instrument Company v. Ans. Inc., 416
F. 2d 943, 945 (2d Cir. 1969) 61/
Appellate courts should be particularly reluctant to substi
tute their judgment for that of the court below where, as
here, the decision rests essentially on factual findings with
respect to discriminatory impact and job-relatedness. These
f indings cannot be disregarded unless clearly erroneous. In
— / United States v. W. T. Grant Co., 345 U.S. 629 (1953);
Societe Comptoir v. Alexander's f)ept. Stores, 299 F. 2d 33,
36 (2d Cir. 1962); Packard Instrument Company v. Ans. Inc., 416
F. 2d 943, 945 (2d Cir. 1969).
Thus where, as here, the court finds the balance of hard
ships tips toward the moving party, he may obtain relief "if
he has raised questions going to the merits so serious, sub
stantial and difficult as to make them a fair ground for
litigation and thus for more deliberate investigation,"
Checker Motors Corp. v. Chrysler Corp., 405 F. 2d 319, 323
(2d Cir.) cert. denied, 394 U.S. 999 (1969); Pino de
Laurentiis Cinematografica, S.P.A. v. D-150, 366 F.2d 373,
375 (2d Cir. 1966).
59
the instant case they were fully supported by the record.
The injunctive order issued was hardly "sweeping" and
"unprecedented"as characterized by the Examiners. It was
carefully drawn on the basis of a series of proposed orders
and supporting affidavits, exhibits and memoranda. It was
clearly reasonable, — / and consistent with orders issued in
comparable cases.
The court's findings as to irreparable injury and the
balance of hardships were clearly correct (See App. 223-25a,
255a). Denial of the injunction would perpetuate racial dis
crimination and deny plaintiffs and their class any opportunity
for appointment throughout the undoubtedly lengthy period until
this case is resolved on the merits; and opportunities for
Its prohibition against use of lists based on examinations
covered by the survey was not disputed by any party below.
Its prohibition against use of lists not yet promulgated when
this action was instituted is obviously reasonable in light
of the court’s refusal to bar further examinations at that
time. And in prohibiting future examinations the court pro
vided defendants full opportunity to obtain a modification
to permit implementation of a new examination system (App. 259a).
Compare, e. q . , the orders issued in Carter v. Gallagher,
supra, 3 CCH E.P.D. f8335, p. 6993-201, and Hicks v. Crown
Zellerbach, 319 F. Supp. 314, 321 (E.D. La. 1970), both bar
ring future examinations until such time as they had been
validated in accordance with E.E.O.C. Guidelines.
60
appointment will then be limited since positions will have
been filled. In addition, hundreds of minority acting
supervisors, selected by local boards on the basis of their
ability to perform, would lose their positions by operation
71/of law if relief were not granted. By contrast, those
persons who are or would be on the eligibility lists have an
equal opportunity, under the court's order, to compete for
positions on an acting basis. Since tenure has been elimi
nated for supervisors they will lose no tenure rights. And
if defendants ultimately prevail on the merits these persons
will have ample opportunity for appointment to all the positions
filled by acting personnel during the interim.
Finally, the court's finding that no harm would result to
— 7 The fact that tenure for supervisors has been eliminated
does not mean as a practical matter that such positions will
be available since traditionally virtually no one has been
discharged even during so-called probationary periods.
71/— * The court found, for example, that there were 131
minority acting Assistant Principals (App. 223a). The
Examiners' argument (Brief p. 45) that only a few people would
be threatened with the loss of their acting positions is ab
surd since it assumes erroneously that the only position in
volved in the instant action is the Elementary School Principal
position. In fact it is clear that every supervisory position
is covered and that there are hundreds of acting minority super
visors in the system.
61
the public (App. 225a, 255a) was obviously warranted by its
conclusion that the Board's examination system was not job-
related. It was further supported by the facts: (1) that the
Chancellor considered the examination system unworkable; (2)
that he and the Board of Education, the officials responsible
for the administration of the City's schools and appointment
of personnel, have chosen not to actively participate at any
stage of the proceedings,^ including this appeal; (3) that
acting appointments have been a tradition in the school system,
constituting a large percentage of total appointments, and (4)
that in the past appointing authorities have depended in large
part on an acting appointment system to obtain the most quali-
fied personnel. Since no other City School System in the
State has an examination system like New York's (App. 184-85a),
it is hard to imagine that much harm would result from its tem
porary suspension, particularly since virtually every group
which has studied the Board of Examiners has concluded that it
should either be abolished or its procedures radically re-
74/formed.
72/— The Board of Education's participation was limited to
opposing the T.R.O. at relatively early stages in the pro
ceedings below; it did not oppose entry of the preliminary
injunction after the court's decision.
— / Indeed the Board of Education had in the past approved
the practice of making acting appointments as a means of in
creasing the number of qualified blacks and Puerto Ricans in
the supervisory ranks. Exhibit 2 to plaintiffs' motion, Ind.
Doc. No. 2. And acting appointments pending development of a
new examination were approved in C.S.A. v. Board of Education.
23 N.Y. 2d 458, 29^ NY Supp. 2d 547, 245 n.2 2d (1969). That
case also held acting principals qualified on the grounds
they had satisfied state certification requirements.
74/ See authorities cited n. 20, supra.
62
Moreover, developments since the court's decision
demonstrate that the decision will not only result in no
harm to the school system but, indeed, may result in the
first significant reform in examination procedures in the
Board's history. in the meantime a workable interim selection
procedure has been devised, which guarantees that acting
supervisors will possess adequate education and experience
qualifications, and will be selected on the basis of merit
in accordance with regularized procedures.
CONCLUSION
the reasons stated above, the district court's
decision granting preliminary relief should be affirmed.
Respectfully submitted,
JACK GREENBERG
JONATHAN SHAPIRO
ELIZABETH B. DUBOIS
10 Columbus Circle
New York, New York 10019
Phone (212) 586-8397
GEORGE COOPER
435 West 116th Street
New York, New York 10027
MICHAEL 0. FINKELSTEIN
26 Broadway
New York, New York
Attorneys for Plaintiffs-Appellees
63
APPENDIX A
Ka y e .Sc h o l e r . Fierman Hays S, Ha n d l e r
•42 5 P A R K A V E N U E N E W Y O R K N Y 1 0 0 2 2
i2\2' P l a z a 9 8 4 0 0
HAROLD l r i fRMAN
MM 1 ON HANDLER
FRLOfW CK R LI/iNCSTON
Stani t y D wAxur nG
MILTON KJNEN
NATHANIEL H JACKSON
R'CHARD C ‘ l FhC h
JOSEPH C* CONNOLLY
Sidney j su b e r m a n
FREDERICK GLIDE WG
JAY O. KRAMER
STANLEY P . ROBINSON
S'DNEY A DIAMOND
FRED A H-i ' A n
NORMAN S'NHMCh
DONLN G i t "*
J O S H U A F G* ; ■ ' H E R
P f TER M c s -- P- N
^ A A S l V . i N t W ' . - A N PETER v.
S L ' M - .? ■ ' - . A • E N
3A u , j o PLAN
DA'. O' GOLL0F «?L M t‘ . 1 Ml
B E " P A M A • - M
■■ ■ r g r j r ■*! i-fr~i~ y~ 1' r n r r r r ^
SAUl l c •* N
MM TON R A t ST I
B W NIM K l N
Wit l 1AM J ISAAC TON
SHCt DON O' M N' IS
FRED N FISHMAN
FREDtRiC.K M DiM.LEN
- GERALD FI l l E R
ARNOLD i . Gin-DDERG
Lfl'IN A : S< MAN
SIDNt ' *■ At L ' t .
S'AN £ I R( . EN S
ROBER’ M SC mAn .’ i r
JAY G STRJM
November 9, 1971-
J A COB SCH O L E R
JAMES S. HAYS
COUNSEL
EUROPEAN OFFICE
4 4 C H A M P S El > SEES
PARIS V»l .FRANCE
TEL i t5 88'0
c a b l e a d p w e s s c s
KAYEMACLEFV n e w YORK
KAYEMAC lER PARiS
y Cl I * NUMBERS
N E W YORK 234060
PARIS 6598i
Elizabeth B. DuBcis, Esq.
10 Columbus Circle, Suite 2030
New York, N.Y. 10019
Dear Miss DuBois:
As I told you i n our telephone conversation of
yesterday, the Board of Examiners has been diligently at
work on a proposal for modifying its supervisory examination
procedures.
findings and
waives none <
believes tha'
fulfill its
it. would be
visory posi
pending t:o!
at it has a n
+- t ■ i t o r* r*
f q i t r H . . oa a -l i L/ O' . * —
tions to U 0 J.
. *_ ... >Y , r- f
ccordingi y ,
t prejudi C c ,
î3.rd sl r » Q p giy d
of T i l , T O LA a Y Po Mans •rie
S by LS Lx rsu ing th i
oi 1 iga1 * c r. as a P
net:on. ■: e E;Car ' YD
j the ent,
ur appeal
A. d . J- I J -L -L
”* 0 0 x am _l i j
not feel
for super-
s y s t em
is to forward to
il for modifica-
.y, the Board has asked
+- o t t c c r o ^ >•* v » T r*r -o <Ty : - D" di- l U L - L d p i p v-y .
tion of supervisory examination procedures. We hope you will
agree that tne proposal. reflects a orairiauic change in the
entire examination approach.
It is cur further hope that this proposal can be
come the basis for a joint application to Judge Mansfield
for a modification of the preliminary injunction.
Very truly yours,
Z. Cohen
9 r7 ( ' • q \
e r. c .
BY HA:
la
BOARD OF EXAMINERS
Proposal for Modification of
Supervisory Examination Procedures
Pursuant to the final paragraph of the preliminary
injunction issued on September 20, 1971 by the Honorable Walter
R. Mansfield of the United Stages District Court for the South
ern District of New York, in Chance, Mercado, et al . v. The Board
of Examiners and the Board of Education of the City of New York,
et a 1., the Board of Examiners is applying for modification of
the preliminary injunction to permit further examinations for
supervisory positions and the promulgation of lists, issuance
of licenses, and making of appointments based thereon. The
application is made without prejudice to all rights on appeal.
It is made on the basis of a proposal for new examination pro
cedures that have been developed during intensive meetings involv
ing the following experts:
Dr. Roscoe C. Brown, Jr., Professor of Education and Director
of the Institute of Afro-American Affairs, New York University
Dr. Harry Gilbert, Professor, School of Education, and
Coordinator of Urban School Psychology Programs, Fordham
University
Dr. Nathan Jaspen, Chairman, Department of Educational
. Statistics, New York University
Dr. Gerhard Lang, Chairman, Department of Educational Research
and Professor of Psychology and Education, Montclair State
College, Montclair, New Jersey
Dr. Robert Linn, Head of the Department of Research and
Development, Educational Testing Service, Princeton, New
J ersey
3a
Dr. Frederick J. McDonald, Chairman, Teacher Behavior Group,
Educational Testing Service, Princeton, New Jersey
Dr. Joseph L. Moses, Psychologist, American Telephone and
Telegraph Company
Dr. Robert D. North, Associate Director, Professional Exam
inations Division, Psychological Corporation, New York,
New York
Dr. Benjamin Rosner, University Dean, Teacher Education,
City University of New York
Dr. Marvin Sontag, Associate Professor of Psychology and
Education, Teachers College, Columbia University
Dr. Robert L. Thorndike, Richard March Hoe Professor of
Psychology and Education, Teachers College, Columbia University
Dr. E. Belvin Williams, Associate Dean for Administration,
Teachers College, Columbia University, (formerly Northeast
Regional Chairman of Association of Black Psychologists)
The proposal concerning new examination procedures
represents the considered professional judgment of the experts
who participated and is intended to gain the widest possible
support for the Board* s procedures and improve their compatibility
with the decentralised school system administration. These experts
not only were given an opportunity to review past examinations
(written tests, oral tests, performance tests, and conference tests),
but they also brought to the deliberations expertise in a variety
of areas: psychometrics in education and in industry, development
and-administration of teacher and supervisory educational prepara
tion programs, analysis of teacher and supervisor behavior in the
school setting, and knowledge of the current status and thinking
of minority groups witti respect to testing in general and testing
for teaching and educational supervisory licenses in particular.
4a
The proposal deals with new procedures in a compre
hensive supervisory license assessment program in four areas:
job analysis; job-related examinations; standards and reporting
of results; validation of examinations.
1. Job Analysis
Proposed Procedure' " " —— y,
The Board of Examiners will request from the Chancellor, in
addition to statements of duties of the position, appropriate
skills, qualities and behaviors, in other words, specific job
analyses for each position for which it is required to conduct
|examinations. Following the consensus of the experts consulted,
the Board of Examiners will develop specific criterion measures
by devising appropriate means of identifying essential job
behaviors and by consulting actual logs of incumbent supervisors
7 of critical incidents on the job as defined by John C. Flanagan.
These will reveal what the person must do on the job, what per
formances are of critical importance and what actions differ-
'entiate between the well-perceived and the inadequate individual
in this job setting. In addition, specific perceptions of the
job will be obtained from Community School Boards, parents and
community superintendents and these will be incorporated into
the criteria for selection of the supervisors for whom the tests
will be given. In addition, research literature which supports
the relationship between specific knowledges and skills and
specific criteria of effective supervisory or administrative
\>a
performance will be consulted to identify further the qualities
to be assessed. This procedure will provide a broader empirical
basis for the job-related examinations to be developed.
2. Job Related Examinations
Proposed Procedure
Job relationship of tests will be established by closely
matching tested knowledge and skills to the essential critical
behaviors required on the job as identified. The language
mastery element in written tests will be deemphasized to take its
place alongside such data as are obtained through other tests
which reveal essential elements of administrative ability, super
visory ability, attitude toward tne learning process for the vary
ing student populations of the city, human relations skills and
attitudes, and such essential additional skills as the candidate
may offer, e.g., knowledge of spoken Spanish, knowledge of Afro-
American and Hispanic culture, etc. The job history and experience
of applicants will be reviewed for relevance to the particular
'license being sought. To illustrate, significant elements of
administrative ability can be tested without recourse to a written
examination by having a candidate defend orally the actions he
would take on a number of realistically simulated documents he
would be faced with in his "in-basket" on the job. Similarly,
important aspects of supervisory ability, that is, the ability
to observe a lesson and to help the teacher improve it, could be
tested by having the applicant observe an actual lesson or a
6a
video taped one, and then evaluate it orally before a panel.
Group interviews may be used to assess an individual's performance
in relation to other members of the group. Video-taped simula
tions of critical incidents involving significant human relations
skills may be used to appraise an applicant's ability to handle
a critical situation. In every instance the test task would be
carefully related to the job behaviors and skills which make up
the criterion derived from job analysis and study of available
research.
3. Standards and Reporting of Results
In the past, specific weights were attached to each one of
the battery of tests with a few exceptions (written English,
physical and medical, and record were rated either satisfactory
or unsatisfactory). A candidate had to achieve a weighted
average of 60 per cent on the battery of tests to pass. If a
candidate secured less than 50 per cent on any part of the exam
ination, he was eliminated. If he obtained 50 per cent or more,
'he was continued in the examination but he had to compensate for
marits lower than 60 per cent because he had to achieve a weighted
average of 60 to pass. In many instances, the written test was
weighted at 40 to 60 per cent of the total grade.
Proposed Procedure
The proposed procedure is made possible by a provision of the
1969 Decentralization law which requires the preparation by the
, a
Board of Examiners of lists of eligible supervisors through "open
qualifying" examinations.
In this procedure, a series of job-related tests will be
retained but the results will be reported in a completely differ
ent manner. A profile of performance will be available to the
Community School Boards for all qualified candidates. This profile
will present an assessment of 'she performance of all candidates
in all of the tests without specific weight and without exact
numerical grades. A sample profile follows on the next page.
STEP
ONE
STEP
TWO
8a
Sample Assessment Profile
(The ter.to indicated are merely 'examp]or. of those to be
used, but total ace or, ament is not restricted to these.)
a;5 C,o Pc Mp H*1—*H* H*0)H* CifD
ft 1Understanding of Essential Principles
of Administrative Behavior
(as measured by "in-basket" situations)
Ability to Evaluate
Observed Classroom Teaching and to
Effect Improvement
(as measured by response to actualteaching)
Understanding of Human Relations Prin
ciples Involved in School Management
(as revealed by reaction to taped
situations)
Understanding of the Educational Pro
cess in the Urban Setting
(as revealed in a written test on edu
cational literature - for entry level
as chairman or assistant principal)
Communications Skills: Ability to
Communicate to Staff, Students, Community
(as revealed in simulated tasks)
Skill in English Communication
(as revealed in a writing sample - for
entry_ level as chairman or assistant principal)
Physical and Medical Test
e>
Record, Including character and evalu
ation of past experience
(including fingerprint check required
by Board of Education)
Special Skills Tested at Applicant's
Request (e.g. knowledge of Spanish) <>
>
Special Experience
(identified from vita supplied bv
applicant and verified by the Board
of Examiners)
Performance on the job
(by team including professionals and
representative of Community School Doarc
at request of Community School Board foi
principals' licenses)
c
DM 1
O 0 1 p
2.1!" :.i •:
v •c» ... l.
O'! ' ;
( :
Qualified
9a
<
It can be secnlrom this sample profile that, though
the candidate is qualified overall, he is unqualified in one of
the areas tested. In the event that the total performance of
the applicant results in a rating of "unqualified" in two or more
areas, he would be rated unqualified and not recommended for
licensing and a place on the qualifying list. Qualifying scores
would be set at levels that would insure minimum competence
necessary to meet the performance criteria on the job. These
would be arrived at after thorough professional review of exist
ing research concerning the skills tested and professional judg
ment as to minimum levels of acceptable performance. Upon receipt
of the list of qualified eligibles, together with their assessment
profiles, each Community School Board could evaluate the indicated
achievement levels of each eligible in the light of its own needs
attaching more importance to some aspects of the total assessment
and less importance to others. Thus, a candidate who did poorly
in English communication but well ir. administration and super
vision might be more acceptable to one Community School Board than
to another. Another aspect of the profile would be the evaluation
of additional skills claimed by the applicant; thus, an applicant
who claimed fluency in Spanish would be tested for that skill and
this additional qualification will be reported together with the
other aspects of his assessment.
The Board of Examiners would undertake to provide a
manual to help Community School Boards interpret the results
10a
reported and would consult upon request with any Community School
Board in matters of personnel selection. Each Community School
Board would then submit to the Board of Examiners a list of quali
fied persons in whom they were interested for principalships for
an assessment of their performance in the job they now held.
(Practical considerations of available experts and cost factors
V
preclude performance evaluation of all eligibles in some exam
inations where there are large numbers of applicants.) The Board
of Examiners, after making such assessment, would evaluate each
eligible as to current performance in the light of criteria
cooperatively arrived at by the Board of Examiners and the
Community School Board involved. Accompanying the profile would
be descriptions of the performance of the qualified applicants
to enable the Community School Boards to have as full a knowledge
as possible of the qualifications of the applicants in question.
Implicit in the assessment program procedures are the same safe
guards that are currently in effect: professional selection and
training of the examination assistants who construct and administer
the individual tests and the right of the applicants to full dis
closure of the guidelines used in the assessment process and the
right to appeal from any or all of the findings.
ty. Validation
Proposed Procedure
The matter of validation is an extremely complex one. This
was recognized by the Court in the extensive discussion of the
11a
problem in its Opinion (pp. 27-35), and in a memorandum
accompanying the preliminary injunction in which the Court urged
that the parties show "a willingness to yield to the public
interest at stake rather than to insist on soecific criteria
that may be impractical or unrealist ic wnetTier~in the field of
predictive validity or otherwise." The difficulties of valida
tion are further illuminated by the decision in V .0. v. Georgia
Power Co., 3 FEP 767 (U.S.D.C.-N.D. Ga. 1971) wherein the Court
observed: "According to the government's experts, it would re
quire at least two years to develop a testing program to meet all
the criteria for validity." 3 PEP at 780. The Court there went
on to note (3 PEP at 7°7 fn.3):
"However, the rather startling evidence offered
by the government was to the effect that there
was no test known to exist or yet devised which
could meet such standards (of the Equal Employ
ment Opportunity Commission). The Court con
cludes that Section l was net intended by
the Congress nor Inter;reted by the Supreme Court
in such meaningless- fashion. • Psychological test
ing is itself a now inexact science. Reputable
able people (her-.-,- two former partners) readily
disagree over the ultimate validity of almost any
test when related to a particular job or employer.
In Grir-ns, validity is a reasonable, not absolute,
requirement. It is inescapable that the Equal
Employment Opportunity Commission regulations,
while on the whole helpful, are not binding in
this respect. See the common sense approach taken
in United States v. H.K. Porter Company, 296 F.
Supp. lid at 7^, 79, 76, 1 PEP Cases 615, 70 LRRM
2131 (N.D. Ala. 1*68)."
With these comments in mind, it is proposed that the
new procedures supplement prior efforts in establishing content
validity with an additional test of validation: an on-the-job
performance evaluation by the Board of Examiners of supervisors
two years after1 they have been in their new job. This performance-
based evaluation would take into consideration (a) the criteria as
to job behaviors previously obtained from supervisors and admin
istrators on the job and (b) feed-back from the Community School
Boards and top-level supervisors as to evaluation of the per-
V.
formance of appointed eligibles. Roth sets of data would then
be compared with the original assessment to result in a valida
tion of these original tests. Such evaluation, would serve two
purposes: to enable validation of the original assessment and
to provide an impartial evaluation of the incumbent’s performance.
This on-the-job performance evaluation would enable follow-up of
significant numbers of licensed supervisors. Evidence obtained
from these performance-based evaluations would make available data
enabling improvement of the assessment program by retaining those
parts which correlated best with performance and eliminating those
which correlated worst.
'Conelusion
The Board of Examiners is presenting this proposal as
the consensus of the experts consulted who evinced a remarkable
degree of agreement. It represents a conscientious effort to
take yet another hard look at its procedures and to make them
more flexible and adaptable to the Community School Board type
of administration. It does so without abandoning the essential
elements of a merit system which protects the rights of all
13a
applicants.
the ultimate
varied comma
Most important it takes cognizance or the needs
beneficiaries of this action, the children in tho
ities of New York City.
V
APPENDIX B
Memorandum for
Subject: Proposal to N.Y.C.
MR. BARRETT
MU. COHi.N
MR. COOPER
MS. DUBOIS
MR. GREENE
MR. SCH/.PIRO
MR. ROCKOVITZ
From: Frr.derick J. McDonald
Date: December 21, 1971
Attached is a copy of the draft of our proposal. After I have
discussed this draft on December 27th, I will prepare a final copy.
I have omitted the budget, but will bring a preliminary draft
to the meeting on the 2 7m. The final draft will include Appendices and
vitae of the personnel who will work on this project.
This draft copy omits points that I think should he discussed among
the parties involved before and if they are to be included in the proposal.
Please keep this in mind as you read the proposal.
14a
15a
PRi: I'ACl.
This Jov unu-nt describes t !.e general methods and worx plan to he
used by ETS to develop a prototype assessment system for the selection of
candidates for an administrative position in the New York City Schools.
It was thought to bo unnecessary to describe these procedures in detail
since they are well known.
It is impossible to predict in this proposal the exact character of
the assessment system to be developed. Rather, the proposal describes
the empirical methods that will he used and out of which an assessment
system will evolve.
The proposal commits the developers, ETS, only to create a system
that is non-discriminatorv and job relevant. There is reason to be
optimistic that the plan described here is likely to produce such a system.
16a
INTRODUCTION
This proposal describes a plan for developing a prototype assessment
system for selected administrative or supervisory examinations to be used
by the Board of Examiners of New York City. The purpose of developing this
prototype is to create a model of an assessment system that is job relevant
and non-discriminatory where job relevance is the primary criterion but
where its application does not systematically or inadvertantly exclude
individuals on the basis of race, sex, religion, and other non-merit factors.
"Job-relevant" means that the assessment procedures will identify those
individuals who have the knowledge, skills, and personal characteristics
which can be shown to be directly related to effectiveness in the position
for which a selection is being made. "Non-discriminatorv" in this instance
means that the assessment system will select or reject members of ethnic
groups in proportion to their numbers in the applicant group.
"Prototype" means an assessment system which may be used as a model
for developing evaluation procedures for selecting individuals for other
positions, and which when approved may be used in the examination system
of the Board of Examiners.
The product of the plan proposed here will be: 1) a description of
the knowledge, skills, and personal characteristics required for the position
for which the assessment system is being developed; 2) a description of
the criteria local boards will use in choosing among candidates for the
position; 3) a set of assessment procedures or techniques; 4) alternative
plans for using these procedures so that they will provide the information
the local boards need to make selections; 5) data on the non-discriminatory,
job-relevance, and other psychometric characteristics of the alternative
procedures; 6 ) cost estimates for the use of the procedures; 7) a set of
17a
recommendations on to!low-up methods to be used to gather data on the
effectiveness of the procedures in selecting successful administrators
or supervisors.
Tasks and Methods
In this section we describe the tasks to be accomplished and the
methods to be used in accomplishing them. hater sections describe the time
schedule of the plan and the personnel who will work on this project.
Job Analyses: Extensive job analyses will be made of the position in
selected school districts. The following methods will be used:
1) ETS staff will follow the incumbents of the position through their
daily routine to determine the nature of their work. A sample of days over
a two month period will he drawn to select the days on which ETS personnel
will be present. ETS personnel will "follow" the incumbent of the position
through his or her daily schedule (with minimum interference with the
incumbent's duties), being present in as many situations as is possible
without interfering with confidentiality in interpersonal relations or
accomplishment of the task at hand.
The person being observed will be fully informed of the nature and
purpose of this activity and will be encouraged to be co-worker of the
observers. They will be asked about their choice of methods to cope with
the problems that arise and the bases of their decisions.
2) An analysis will be made of all the correspondence, telephone calls,
and face-to-face requests made of the incumbent. When these materials and
conversations are confidential, the incumbent will be asked to summarize
thei r content and his de • i sLons aid act ions.
3) The incumbents wi11 he .skec to keep a Ion of t lie i r activities
duri ng the periods when ETS staf:: are not present.
18a
and
4) An analysis will bo made bv KTS personnel of the legal, policy,
procedural statements governing the position beins studied.
5) ETS personnel will make an analysis of documents describing what
the position "ought" to be.
6 ) ETS personnel will interview teachers, community superintendents,
community board members, and representatives of major educational organizations
such as organizations of school administrators and parents' groups about
theii perceptions of what the position requires and the characteristics
the incumbent ought to have.
A description of the position will be drawn from these observations
and analyses. This description will include a list of the tasks the incum
bent must successfully accomplish, the decisions he must make, the characteristic
actions or methods he must use, the knowledge, skills, and personal traits
presumed to be required to accomplish the tasks, make the required decisions,
and successfully execute the actions needed to solve the problems the
incumbent will face.
This description will be used to specify the characteristics of the
assessment system to be developed.
— -h — Develoj^ment^: It is impossible to specify in this proposal the exact
nature of the testing techniques that will be used and the assessment system
that will be proposed. It is expected that the techniques will include
a wide range of procedures such as paper and pencil tests and performance
tests. It is to be understood^ however, that greater effort will be given
to the development of techniques that measure, ability to perform effectively
i_rwlyc_ • it should also be understood that the system developed
will use a variety of procedures w h i c h will he included only if they can
be shown to be correlated with ef-'cct ivt i,ss in the position.
19a
The assessment system will nr u, ,ci,. k
11 ,robably be a set o i successive steps
through which the candidate willwiu pass. His performance on each phase
will be appropriately weighted; and his weighted scores will kscores will be presented
in a profile.
l e ^ L j r ^ O u t - . Each of the procedures developed will be administered
to a sample of candidates. The samples will be determined in the course
tHlS Pr°ieCt‘ Ir may be assumed that the size of these
samples will vary; for example, if a paper and pencil test is developed
first phase, a large number of candidates will take it.
“~ ~ : lhe Pr°Cedures developed will be validated by testing
a sample of current incumbents of the position who will be rated on
I SUCCeSSfUl j°b perfo™anee. Performance on each procedure will be
correlated with measures of on-the-job performance.
Advisory Beard
ETS wi n request individuals knowledgeable about the position and
Che Pr"ble“ °£ Vork «flr schools to serve in an advisory eapaoin
P icet stafi Thrs Advisory Board will recommend problems to be
anticipated and procedures to be followed in the development of the
assessment system. The Board will review progress and final reports.
The advisory Board win meet in the first month of the project. At
this time the Board will be presented the sampling plan for selecting
districts and schools within which the job analyses w i n he made, a des
cription of the work plan for condoning the Job anaiyses, and a description
of the staff and an opportunity to , horn. The Board w i n ho asked to discuss
the problems to be . lf A ...
L ....."citing the job ana lyses and interpreting
the information Lh.it w i l , be gathered.
2 0a
At this i.ine the 5 lan and stafi for interviewing the Community
Superintendents and Boards will also be presented to the Advisory Board
for their review and advice.
The next meeting of the Advisory Board will be held after the job
analyses and interviews with Community Superintendents and Boards have
been held. LI'S personnel will present summaries in the form of job des
criptions which state the performances required of th. incumbents of the
position for which the assessment system is being prepared. The function
of the Board will he to make a critical analysis of th.is description. At
this time EiS will also present the specifications for the assessment
system which will be reviewed by the Board.
« The Board will meet for the third time after pilot forms of the
components of the assessment system have been developed. The Board will
review these components with specific attention to their relation to the
job descriptions developed in the first phase of this project. The Board
will be used as a panel of experts to judge the content validity of the
components 01 the assessment system.
The last meeting ol the Board will take place when a preliminary draft
of the final report is available. The Board will be asked to react to
this report by criticizing the analysis and interpretation of the data
and the recommendations which will have been made.
The above list of tasks to be undertaken by the Advisory Board suggest
the criteria for the selection of its members. The Board must have technical
expertise in assessment and evaluation, familiarity with the problems of
administering New York City Schools, sensitivity to the social, political,
am! r’jlt’ir',i ! ' r f * ‘a1 • •mmtnlt i«..s and >• •..•>'Is . • New York City, and
commitment to helping to produce an assessment system which is non-discriminatory
and job relevant.
21a
The Role of the Community Boards
We have provided for extensive consultation with the Community
Boards and their Superintendents. This consultation will serve three
purposes.
First, the Boards and Superintendents will describe the performances
and characteristics required for the position being studied as they perceive
them. The members of the Boards will be asked what the incumbent should
be able to do, and what they see are desirable characteristics for the
incumbent to have.
Second, when the desired performances have been described by the
processes described above, the Boards and Superintendents will be asked to
react to these descriptions and to weigh their relative importance. A
description ol the criteria and their relative and comparative importance
will be prepared from this analysis.
Third, the Boar.Is and Superintendents will be interviewed to determine
what information they need to assess the potential of a candidate for a
position in their description. They will be asked to review the instruments
developed and their scoring procedures to determine if they are likely to
provide the information required. From this analysis a description of the
information system to be used with the assessment system will be developed.
Construction of Test Procedures
ETS personnel will construct the specific procedures to be used. F.TS
has developed a written examination for Chicago School District which meets
the requirement of being »on-discriminatorv. A copy of a report on this
examination is appended to this proposal fsee Appendix B). Materials similar
to this tyamin.it . • n will be do.eloped, it is understood t nut ETS will not use
tests developed t i other c t ten ts vitnout revising them to make them relevant
to positions in New York City.
2 2 a
E1S has also cIcvl loped many Ji* t«. rent forms o 1 a performance measure
called an in -basket" tost, from trie tray on the desk of an administrator
in which is deposited the incoming mail. This test is a simulation procedure
in which the candidate assumes the role of an administrator in a prescribed
setting and the candidate reacts to "real" problems. His decisions and the
way he carries them out, as if he is actually on the iob, provide a sample
of his administrative behavior. This technique has been used widely in business
and government and performance has been shown to be related to administrative
effectiveness.
The Teacher Behavior Research Group of F.TS has developed simulation tests
of supervisory skills. In these tests supervisory personnel watch videotapes
of classroom teaching on which thev comment. They are scored on what they perceive
as deficient or commendatory, and on the recommendations they would make.
These are some of the kinds of procedures likely to appear in the proto
type of the assessment system to be developed. Technical analysis will determine
what will be included in the prototype to be proposed.
Alternative Systems and Their Costs
It is likely that more than one system can be developed. These alternatives
will differ in the components included, their relative weights, their predicative
validity, their relative utility lor different purposes, the order in which the
components are organized and their comparative costs. We will propose several
such alternatives, relevant cost informal; ion, ana their comparative utilities.
We regard the choice of a prototype for use as an optimizing decision,
that is, one in which the benefits to be achieved are balanced against a
variety of costs. We will provide the information necessary to make this kind
of a decision.
Importance of the Empirical Approach
to the Development of the Prototypes
Many individuals have ideas about what characteristics an assessment
system should have. We are committing ourselves to producing an assessment
2 3a
system which has two characteristics: it will be non-ciiscrimnatorv and job
relevant. Any other characteristics that it: may have will be determined by
empirical analysis.
We are not committing ourselves in advance to any form of testing, though we
suspect that a system to have validity will probably have a major component of per
formance testing. We will provide exemplars of systems likely to have high valid
for selecting the most competent individuals whatever form that system may take.
Work Schedule
A nine month time line is projected for this project. Work on the project
will begin when the contract is signed and registered. Since this date is
presently indeterminate, the schedule below describes the work plan in monthly
modules.
1st Month:
1. Selection of the districts and schools in which the job analyses will
be conducted and whose Boards and Superintendents will be interviewed.
2. Organization of the job analysis plan
3. Organization of the interview schedule to be used with Community
Boards and Superintendents.
4. Organisation of job analysis and interviewing teams.
5. Selection of Advisory Board.
6 . First meeting of the Advisory Board.
7. Development of criteria for evaluating incumbents in the position.
2nd -_3rd Months :
1. Conduct of job analyses.
2. Conduct of interviews with Community Boards and Superintendents.
3. Preparation of job descriptions.
4. Preparation of specifications for testing procedures.
5. Rating of incumbents in the position.
24a
At.h Month:
1. Review of job descriptions ' v Community Boards and Superintendents.
2. Review of job descriptions and test speoificat ions by Advisory Board.
3. Development of testing procedures.
5th - 6th Months:
1 . Development of testing proce dures.
1
2 . Review of testing procedures b v Co mm un i t v Bo a r d s and Superintendents.
t
3. Review of testing procedures by Advisory Boards.
4. Pilot testing of instruments •
7th - 8th Month.:
1 . Analysis by Communitv Boards and Superintendents of their information
needs in selecting potential candidates.
2. Development of information system to be used in conjunction with the
assessment system.
3. Data-gathering for validity studies.
4. Analysis of data for validity studies.
9th Month:
1. Draft of final report.
2. Review of draft of final report by Advisory Board.
3. Preparation of final report.
The final report of this project will be presented within two weeks
after the end of the ninth month.
Personnel
Vitae of project personnel who are currently on ETS's staff and who
will be working on this project are in Appendix C. The amount of time
allocated to this project by these staff members is specified in the Project
Budget. Additional staff such as research assistants will be hired or selected
from the ETS staff.
i