Chance v. Board of Examiners Brief for Plaintiffs-Appellees

    UNITED STATES COURT OF APPEALS
FOR THE SECOND CIRCUIT 

NO. 71-2021

BOSTON M. CHANCE, LOUIS C.
MERCADO, et al..

Plaintiffs-Appellees, 
against

THE BOARD OF EXAMINERS,
Defendant-Appellant,

and
THE BOARD OF EDUCATION OF THE CITY 
OF NEW YORK, et al..

Defendants.

Appeal From An Order Of The United States District 
Court For The Southern District Of New York

BRIEF FOR PLAINTIFFS-APPELLEES

JACK GREENBERG
JONATHAN SHAPIRO
ELIZABETH B. DUBOIS 

10 Columbus Circle 
New York, New York 10019 
Phone: (212) 586-8397

GEORGE COOPER
435 West 116th Street 
New York, New York 10027

MICHAEL O. FINKELSTEIN 
26 Broadway 
New York, New York

Attorneys for Plaintiffs-Appellees



TABLE OF CONTENTS

*1

Statement of The Issues Presented for Review 
Statement of The Case ................

(1) The Proceedings Below ..............
(2) The District Court's Decision .....
(3) The Preliminary Injunction .........
(4) Subsequent Developments ............

ARGUMENT:

PAGES
1
2
2

12
17
18

Introduction .......................
Point I. The District Court's Finding That

Defendants' Examination System Had A 
Significant And Substantial Discrimi­
natory Impact Was Clearly Supported By The Record ..........................

Point II. The Court Below Correctly Ruled That A 
Public Employer Violates The Equal 
Protection Clause In Conducting A Pro­
motional Examination System Which 
Systematically And Significantly Dis­
criminates Against Ethnic Minority Groups 
Where That Examination System Cannot Be 
Shown To Have Any Relation To Job Per­formance ........
A. A Prima Facie case Of Discrimination Is 

Made Out Where Examinations Significantly 
And Systematically Exclude Members Of 
An Ethnic Minority Group Regardless Of 
Whether Any Subjective Intent To Discrim­inate is Shown ..........

B. Where Employment Tests Are Shown To Have 
A Significant And Substantial Discrimi­
natory impact On Ethnic Minority Groups 
They Violate The Equal Protection Clause 
If They Cannot Be Shown To Be Job-Rela-

(1) The Court Applied The Proper 
Standard In Determining That An 
Adequate Case Of Discriminatory Impact Had Been Made Out .......

i



PAGE

POINT III.

(2) The Court Properly Determined That 
Examinations Which Have Such A Discrimi­
natory Impact Violate The Equal Pro­
tection Clause If They Cannot Be Shown 
To Be Job-Related ................  a.4

The Court's Finding That Defendants'
Examinations Had Not Been Shown To Be

4 And Apparently Were Not Job-Related Was 
Clearly Supported By The Record ............ 52

v POINT IV. The Grant Of Preliminary Relief, Based 
Upon The Court's Findings With Regard 
To irreparable Injury And The Likeli­
hood Of Success On The Merits, Must Be 
Upheld As A Proper Exercise Of Dis­
cretion .....................  50

Conclusion ...
Appendix A ...
Appendix B ...

4

1 1



TABLE OF AUTHORITIES

Cases
Page

Armstead v. Starkville Mun. Sep. School Dist.,
325 F. Supp. 560 (N.D. Miss. 1971)......  23,37,44,49,57

^Arrington v. Massachusetts Bay Transportation 
Authority, 306 F. Supp. 1355 (D. Mass.
1969) ....................................... 36,45,49, 57

Baker v. Columbus Municipal Separate School
District, 329 F. Supp. 706 (N.D. Miss. 1971) .. 37,44,45,

49,57
Bolling v. Sharpe, 347 U.S. 497 (1954) ................  47

Brown v. Allen, 344 U.S. 443 (1953) ...................  47

Carmichael v. Craven, No. 26,236, 9th Cir.,
Nov. 4, 1971 ....................................... 35

Carter v. Gallagher, 3 CCH E.P.D. 18205 (D. Minn.
1971), aff’d in pertinent part. 3 CCH E.P.D.
18335 (1971), aff'd in pertinent part en banc.
No. 71-1181 (8th Cir., Jan. 7, 1972) ...........36,39,41,

48,57,60
^Castro v. Beecher, 4 CCH E.P.D. 17569, clarified.

4 CCH E.P.D. 17570, -judgment modified. 4 CCH
E.P.D. 17589 (D. Mass. 1971) ..................  36,40,46

Chaney v. State Bar of California, 386 F.2d 962
(9th Cir. 1967) ...................................  44

Checker Motors Corp. v. Chrysler Corp., 405 F.2d 319
(2d Cir.), cert, denied. 394 U.S. 999 (1969) ....  59

Council of Supervisory Associations v. Board of 
Education, 23 N.Y.2d 458, 297 N.Y.S.2d 547,
245 N.E. 2d 204 (1969) ................. ...... 22 62

Dandridge v. Williams, 397 U.S. 471 (1970) ............ 48

Dino de Laurentiis Cinematografica, S.P.A. v. D-150,
366 F . 2d 373 (2d Cir. 1966) .................. [___ 59

Gaston County v. United States, 395 U.S. 285 (1969)

iii

35



Goodwin v. Wyman, 330 F. Supp. 1038 (S.D. N.Y. 1971) .. 48
Page

Graham v. Richardson, 403 U.S. 365 (1971) ............. 47

Gregory v. Litton System, 316 F. Supp, 401 (Cent.
D. Cal. 1970) ...................................... 40

Briggs v. Duke Power Co., 401 U.S. 424 (1971),
reversing, 420 F.2d 1225 (4th Cir. 1970) 34,37,38,39,42,

47,48,49,57
Harper v. Virginia Board of Elections, 383 U.S.

663 (1966) ..................... ................... 47

Hawkins v. Town of Shaw, 437 F.2d 1286 (5th Cir. 1971) 35,48

Hicks v. Crown Zellerbach Corp., 319 F. Supp. 314
(E.D. La. 1970) ................................  37,57,60

Hobson v. Hansen, 269 F. Supp. 401 (D. D.C. 1967), 
aff'd sub nom. Smuck v. Hobson, 408 F.2d 175 
(D.C. Cir. 1969) ...............................  35,42,48

Jackson v. Godwin, 400 F.2d 529 (5th Cir. 1968) ......  35,48

Jackson v. Wheatley School District, 430 F.2d 1359
(8th Cir. 1970) .................................... 23

Johnson v. New York State Education Department,
449 F. 2d 871 (2d Cir. 1971) ......................  48

Johnson v. Pike Corp., 4 CCH E.P.D. J7517 (Cent.
D. Cal. 1971)   37,40

Jones v. Georgia, 389 U.S. 24 (1967) ..................  41,47

Kennedy Park Homes Ass'n v. Lackawanna, 436 F.2d
108 (2d Cir. 1970) ................................  35,47

Kirkpatrick v. Preisler, 394 U.S. 526 (1969) .......... 41

Korematsu v. United States, 323 U.S. 214 (1944) ......  47

Kramer v. Union Free School District, 395 U.S. 621
(1969)   46

Loving v. Virginia, 388 U.S. 1 (1967) .................  47

IV



Page
McLaughlin v. Florida, 369 U.S. 184 (1964) ............ 46,47

Norris v. Alabama, 294 U.S. 587 (1935) ................  46

North Carolina Board of Education v. Swann,
402 U.S. 43 (1971) ................................  35

Packard Instrument Company v. Ans. Inc., 416 F.2d
943 (2d Cir. 1969) ................................  59

Patton v. Mississippi, 332 U.S. 463 (1947) ............ 47

Penn v. Stumpf, 308 F. Supp. 1238 (N.D. Cal. 1970) ___  36,39

Pickens v. Okalona Mun. Sep. Sch. Dist., No.
EC6956-K (N.D. Miss., Aug. 11, 1971) (mimeo.
OP-) ...............................................  46,49

^orcelli v. Titus, 431 F.2d 1254 (3d Cir. 1970),
aff1 g, 302 F. Supp. 726 (D. N.J. 1969) ........... 22,23

Powell v. Power, 436 F.2d 84 (2d Cir. 1970) ........... 35

Robinson v. Lorillard Corp., 444 F.2d 791 (4th Cir.
1971) ........................................ 37,45,46, 50

Schware v. Board of Bar Examiners, 353 U.S. 232 (1957) 44

Shapiro v. Thompson, 394 U.S. 618 (1969) ...............  47

Sims v. Georgia, 389 U.S. 404 (1967) ...................  41

Smith v. Texas, 311 U.S. 128 (1940) ....................  35

Societe Comptoir v. Alexander's Dept. Stores, 299 F.2d
33 (2d Cir. 1962) ..................................  59

Southern Alameda Spanish Speaking Org. v. Union City,
424 F . 2d 291 (9t.h Cir. 1970) ....................... 35

Turner v. Fouche, 396 U.S. 346 (1970) .................  41

United States v. Bethlehem Steel, 446 F.2d 652
(2d Cir. 1971) ..................................... 46,50

United States v. Jacksonville Terminal Co., 3 CCH
E.P.D. f8324 (5th Cir. 1971) ..................  45, 50,57

v



United States v. W. T. Grant Co., 345 U.S. 629 (1953) . 59

United States ex rel. Chestnut v. Criminal Court of
City of New York, 442 F.2d 611 (2d Cir. 1971) ___  41

Wells v. Rockefeller, 394 U.S. 542 (1969) ............. 41

W e s t e r n  Addition Community Organization v. Alioto,
330 F. Supp. 536 (N.D. Cal. 1971) ........... .. 36,40,46

Whitus v. Georgia, 385 U.S. 545 (1967) .................  41

 ̂ Statutes and Regulations

California Fair Employment Practices. CCH E.P. Guide
120,861 ............................................  43

Colorado Civil Rights Commission Policy Statement on 
the Use of Psychological Tests. CCH E.P. Guide 
121,060 ............................................  43

EE0C Guidelines on Employee Selection Procedures.
29 CFR §1607, 35 Fed. Reg. 12333 (Aug. 1, 1970) 43,57,60

N.Y. State CONST. Art. V §6 ............................  44

N.Y. State Educ. Law §2569(1) (1967) ..................  44

' OFCC Guidelines on Employee Selection Procedures.
35 Fed. Reg. 19307 (Oct. 2, 1971) ................  43

V

Pennsylvania Guidelines on Employee Selection Proce-
dures, CCH E.P. Guide 15194 ....................... 43

42 U.S.C. §1981 .........................................  2

42 U.S.C. §1983 .....................................   2

42 U.S.C. §2000-e .................... „

Page

vi



Ot h e r  A u t h o r 1 t i e s

ANASTASI, PRINCIPLES OF PSYCHOLOGICAL TESTING (3d
Ed. 1968) .........................................  56, 57

Center for Field Research and School Services, New
York University, A Report of Recommendations in
the Recruitment. Selection, Appointment and
Promotion of Teachers in the New York City
Public Schools (1966) ............................  24

COLEMAN, EQUALITY AND EDUCATIONAL OPPORTUNITY (1966) . 42

Cooper and Sobol, Seniority and Testing under Fair
Employment Laws. 82 HARV. L. REV. 1598 (1969) ... 42

CRONBACH, ESSENTIALS OF PSYCHOLOGICAL TESTING
(3d Ed. 1970)...................................... 54

Commission on Human Rights for the City of N. Y.,
Egual Employment Opportunity and the New York 
City Public Schools, An Analysis and Recommenda­
tions Based on Public Hearings Held January 25-29.
M ........................... ..................  25

CRESAP, McCORMICK and PAGET, Summary Report of
Assignments Conducted for the New York City
Board of Education (1962) ........................ 24

FISS, A Theory of Fair Employment Laws. 38 U. CHI.
L. REV. 235 (1971) ................................  24

GHISELLI, THE VALIDITY OF OCCUPATIONAL APTITUDE
TESTS (1966) ...................................... 42

GRIFFITH REPORT, Teacher Mobility in New York City:
A Study of Recruitment. Selection, Appointment.
and Promotion of Teachers in the New York City
Public Schools (1963) ............................  24

KIRKPATRICK, et al., TESTING AND FAIR EMPLOYMENT (1968) 42

Mayor's Advisory Panel on Decentralization of the 
New York City Schools, Reconnection for
Learning (1967) [BUNDY REPORT]..................... 24

Mayor's Committee on Management Survey, Administra­
tive Management of the School System of New York 
City (1951) ["STRAYER AND YAVNER REPORT"J....  24

Page

vii



Page
Note, Discriminatory Merit Systems; A Case Study of

the Supervisory Examinations Administered by the 
New York Board of Examiners, 6 COLUM. J. L. &
SOCIAL PROBS. 374 (1970) ......................... 5,22

ROGERS, 110 LIVINGSTON STREET (1968) .................  24

SCHINNERER, A Report to the New York City Education
Department (1961)   24

STAHL, Public Personnel Administration (5th ed. 1962) 56

THORNDIKE, PERSONNEL SELECTION (1949)   54,56

THORNDIKE and HAGEN, MEASUREMENT AND EVALUATION IN
PSYCHOLOGY AND EDUCATION (1969) .................  56

viii



UNITED STATES COURT OF APPEALS 
FOR THE SECOND CIRCUIT 

NO. 71-2021

BOSTON M. CHANCE, LOUIS C. 
MERCADO, et al.,

Plaintiffs-Appellees, 
-against-

THE BOARD OF EXAMINERS,
Defendant-Appellant,

and
THE BOARD OF EDUCATION OF THE 
CITY OF NEW YORK, et al..

Defendants.

Appeal From An Order Of The United States District 
Court For The Southern District Of New York

BRIEF FOR PLAINTIFFS-APPELLEES

Statement of the Issues Presented for Review

1. Whether the district court's finding that the supervisory 
examination system administered by defendants had a significant 
and substantial discriminatory impact upon blacks and Puerto Ricans 
must be upheld as not clearly erroneous?

2. Whether the district court was correct in ruling that 
examinations which have such a discriminatory impact violate the



Equal Protection Clause of the Fourteenth Amendment if they 
cannot be shown to be job-related?

3. Whether the district court's findings that the supervisory 
examinations administered by defendants had not been shown to
be and apparently were not job-related must be upheld as not 
clearly erroneous?

4. Whether the district court's grant of preliminary relief 
must be upheld as not constituting an abuse of discretion in light 
of its findings with respect to irreparable injury, the balance
of hardships and the probability of plaintiffs’ success on the 
merits?

Statement of the Case
(1) The Proceedings Below

J JPlaintiffs brought this class action in September, 1970, 
pursuant to 42 U.S.C. §§1981 and 1983, challenging the examinations 
used to select principals and other supervisors in the City School 
System. Plaintiffs charged that these examinations operated in 
violation of the Equal Protection Clause of the Fourteenth Amendment 
in that they discriminated against blacks and Puerto Ricans and 
could not be justified as job-related. Named as defendants were 
the Chancellor, Dr. Harvey B. Scribner, the Board of Education 
and the Board of Examiners. Plaintiffs moved simultaneously for

--/ The named plaintiffs included numerous blacks and Puerto Ricans
who had failed previous supervisory examinations, who possessed all 
the educational and experience qualifications established for various 
different supervisory positions they sought, but who could not receive 
regular appointments because they lacked the requisite license 
(App. 139a, 143-45a).

2



a preliminary injunction preventing defendants from administering 
the Elementary School Principals' Examination scheduled for 
November 3, 1970 and from giving other supervisory examinations 
in the future until they had been validated.

Plaintiffs documented their claim of discrimination with 
undisputed evidence that blacks and Puerto Ricans were grossly 
underrepresented in the supervisory ranks and on supervisory lists 
as compared to the student population (App. 147-49a). in addition, 
comparative statistics were introduced from other city school 
systems not subject to New York's unique examination procedures, 
including Rochester's, where candidates must satisfy New York 
State certification requirements. These showed New York City to 
have by far the lowest percentage of black and Puerto Rican
supervisors, and the lowest ratio of minority group supervisors

2 /
to minority group students (App. 197-98a). And finally the
statistics showed there was a much higher percentage of blacks 
and Puerto Ricans in acting supervisory positions, where ability 
to perform is the sole criterion of selection, than in licensed 
positions (19% of acting principals as against 1% of licensed 
principals). Plaintiffs also documented the culturally biased 
nature of the examinations themselves, submitting expert evidence, 
based on an analysis of the most important examinations given

_2y Thus Rochester has a black student population comparable to 
New York's (30% as compared to 34%) but 11 times as many black principals.

3



by defendants in recent years, that they were likely to discrimi­
nate unfairly against blacks and Puerto Ricans (App. 35-39a; 
see also App. 149-50a).

Evidence was also introduced indicating that defendants' 
supervisory examination procedures could not be justified as 
job-related. Thus plaintiffs showed that defendants had never 
performed a job analysis - concededly the first and essential step 
in developing a job—related examination — for any supervisory 
position. indeed the first step of the first job analysis plan 
in the history of the New York School System had just been 
undertaken in the form of a research proposal to discover the 
qualities needed by an Elementary School Principal, the very
examination scheduled for November 3, 1970, which plaintiffs

. . . !/sought to enjoin. Plaintiffs also charged that the persons 
responsible for developing and administering defendants' 
examinations had no background or training in the testing field, 
were selected on an ad hoc basis from the teaching and supervisory 
ranks without having to satisfy any special qualifications or 
selection procedures, or being provided any special training in 
test construction and administration and, in addition, that they 
were overwhelmingly of the white race. Finally plaintiffs showed 
that no validity studies had ever been conducted indicating

-17 APP- Ex- 19; DuBois Affidavit, Ind. Doc. No. 44, paras.
7 and 8 . Documents not included in the Appendix or Appendix 
Exhibit Volume will be referred to by their number on the Index to the Record on Appeal.

4



that defendants' examinations were in any way related to job
_4J

performance.
Plaintiffs also submitted evidence that authorities responsible 

for selecting supervisory personnel had found no correlation between 
possession of a license and ability to perform the job at issue, 
that many licensed candidates had been found unqualified to serve 
in the position for which they were licensed, and that in order to 
select the most qualified candidates it had often been necessary to 
appoint persons without the requisite licenses on an acting basis 
(App. ll-34a).

After obtaining numerous extensions of time defendants finally 
filed reply papers October 27, 1970, one week prior to the 
scheduled November 3 Elementary School Principals' Examination.
On the basis of oral argument October 27 and 29, the court 
decided it had not had sufficient opportunity to consider the 
merits of the case before it and therefore refused to enjoin 
the administration of the November 3 and other future examinations. 
However, to maintain the status quo pending consideration of 
plaintiffs' motion, the court granted a temporary restraining 
order preventing the promulgation of new eligibility lists based 
on such examinations (App. 101a).

4/ App. 15a, 12a, 28-29a, 150a, 37-39a. See generally Note, Dis­
criminatory Merit System: A Case Study of the Supervisory Examinations
Administered by the New York Board of Examiners, 6 Colum. J. Law and 
Social Problems 374, 377-88 (1970), Exhibit 13 to DuBois Affidavit,
Ind. Doc. No. 44.

5



The Board of Education took the position at this time that 
the complaint raised issues of fact which should be determined at 
trial, and that it would leave to the Board of Examiners the task 
of defending their examination procedures (App. 99-100a). The City 
School System's chief administrator, Chancellor Scribner, had 
previously indicated that he would "prefer not to defend" against 
the action since "[t]o do so would require that I both violate 
my own professional beliefs and defend a system of personnel 
selection and promotion which I no longer believe to be workable," 
a system inconsistent with the selection of "the most talented 
supervisors; the most able principals and other administrators who 
possess the highest level of leadership qualities possible. . . . "  
(App. Ex. 18, p. 4). Neither the Board of Education nor the 
Chancellor has actively participated in the defense of this action 
to date.

The Examiners argued in their reply papers that plaintiffs 
had not made out a case of discriminatory impact because they 
had not submitted "the only meaningful statistic" —  to-wit, 
"̂ .comparison of the pass-fail ratio of black and Puerto Rican 
applicants." They argued further that

"[i]f such an evaluation were undertaken, 
it would demonstrate clearly that eligible 
black or Puerto Rican applicants have per­
formed comparably with white applicants on examinations." 5 /

_5_/ (Affidavit of defendant Examiner Jay E. Greene App. 87-88a) 
(emphasis added). in addition, Mr. Greene cited a few examples of 
a few examinations in which "It is believed that" blacks and Puerto 
Ricans performed comparable with whites (App. 88-90a).

6



The Examiners defended the validity of their examinations 
at this time on the basis of a number of alleged research and 
validity studies which they did not submit. Thus defendant 
Examiner Rockowitz claimed there were numerous research studies 
which were designed "to determine the validity" and which "confirmed 
the validity" of the examinations (App. 63a); that there had been 
at least three "empirical validity" studies, and that their 
results "indicate substantial positive correlations between the 
purposes of the examinations involved and the results obtained. . . 
(ftPP• Ex. 10, pp. 3—4) (emphasis added). Specifically, he relied 
on the results of an alleged predictive validity study of 
elementary school principals' examinations as showing that 
principals identified as the best performers on the job "score 
higher on the . . . examinations than do a random selected group 
of principals" (App. Ex. 10, p. 4). And finally he alleged that 
the Examiners had conducted 22 "validity studies" and had 
published three volumes relating to research (App. Ex. 10, p. 5).
The Examiners also relied on the opinions of four experts whose 
opinions were based on statements made to them by Board representa­
tives describing examination procedures and on the research 
studies cited by Dr. Rockowitz, together with other documents 
submitted to them. These experts claimed only that on the basis 
of what they had seen and been told the Board appeared to have 
made certain efforts to design content-valid examinations.

term
_6_/ The/predictive validity, used by the court below, is some­
times referred to as "empirical" or "criterion related" validity.
A brief discussion of the meaning of predictive and content validity,
and significance in showing job-relatedness, appears atpp • Orr j o ; m r r a .

- 7 -



1

In response plaintiffs submitted extensive additional 
evidence that defendants' examinations could not be shown 
to be job-related. Recognized experts, including Dr. Enneis, 
staff psychologist in the Office of Research of the U. S.
Equal Employment Opportunity Commission, concluded 
that the documents entitled "Duties of the Position," which 
were the only job descriptions used by the Examiners in con­
structing examinations, did not constitute job analyses and 
were an inadequate basis for developing a job-related selection 
system (App. 108-09a, 117a). They also concluded that the 
Examiners' experts had not conducted the kind of analysis and 
investigation essential to support a claim of content validity 
(App. 112-114a); and that in any event predictive validity 
studies were both feasible and, indeed, were essential to 
establish job-relatedness for the type of examinations administered 
by defendants (App. lll-12a, 114a, 116a). Plaintiffs intro­
duced additional evidence documenting previous charges that no 
job analysis had ever been conducted, and that the assistant 
examiners who designed and administered examinations had no 
special background or training (App. 129a, 131-32a, App. Ex.
14) .

At a hearing November 19, 1970, the court indicated that 
comparative statistics as to actual pass rates of different 
ethnic groups were essential to assess the discriminatory impact 
of defendants' examinations. Since plaintiffs had no access
to such information and since defendants claimed they had no

8



racial records, the court ordered the parties to develop a 
survey procedure to determine comparative pass rates on a 
significant sample of examinations.

Plaintiffs asked that defendants be ordered to produce 
statistics on the pool of eligibles, as well as on applicants, 
so as to document the charge that defendants' discriminatory 
examination system had a chilling effect, discouraging blacks 
and Puerto Ricans from even applying. Defendants had claimed 
that no such statistics were available, and the court refused 
to order them produced on the grounds that the essential issue 
in determining discriminatory impact was the effect of the 
examinations upon actual applicants (Ind. Doc. No. 25, TR 10-13, 16).

Elaborate survey procedures were worked out by the parties 
and the court and incorporated in four different court orders.
The survey covered the most important supervisory examinations 
given in the past seven years and all supervisory examinations 
given in the past three years. Over 6000 applicants were in­
volved. The survey was conducted by the Board of Examiners and 
the Board of Education with the assistance of the National Opinion 
Research Center, and took several months to complete. Following 
the submission by both sides of expert affidavits (Ind. Doc. Nos. 
61-63, 66) and briefs (Ind. Doc. Nos. 60, 64) directed to the 
relevance of the statistical tabulations, an evidentiary hearing 
was held May 21, 1971 (Ind. Doc. No. 26), to clarify apparent 
conflict between the parties' experts. At this hearing plaintiffs 
presented essentially uncontradicted expert evidence that the

7/ App. 189a, 231a n. 14; Ind. Doc. Nos. 11, 15, 23.

9



statistics revealed that defendants' examinations had a significant 
and substantial discriminatory impact upon blacks and Puerto Ricans.

At the conclusion of the hearing the court afforded both 
sides an opportunity to provide further evidence by affidavit 
or oral proof respecting the issue of job-relatedness (TR 110,
Ind. Doc. No. 26).

Plaintiffs submitted all the Examiners' alleged research 
and validity studies which were obtained upon court order pursuant 
to a discovery request, Ind. Doc. No. 17. Plaintiffs also submitted 
an extensive expert analysis of these studies in the form of an 
affidavit submitted by Dr. Richard Barrett (App. 155-64a, App.
Ex. No. 15), a recognized authority on employment testing and 
discrimination (App. 35-36a). Dr. Barrett stated that he had 
examined all the alleged research and validity studies relied 
upon by Dr. Rockowitz in his original affidavit for his claim 
that the examinations were job-related. Dr. Barrett concluded 
that no study supported a claim of predictive validity and no 
study related to a claim of content validity. He stated further 
that the only study even relating to validity, entitled Analysis 
of Examination for Principal of Elementary Schools (Exhibit 1 to 
Barrett Affidavit, Ind. Doc. No. 68), indicated that there was in 
fact no relationship between the examinations in question and 
job performance (App. 159-61a). The remaining research studies 
consisted solely of studies dealing with the internal structure 
of tests, the research proposal for the development of a job 
analysis for the Elementary School Principal position referred

10



to supra, p. 4, descriptions of teacher selection procedures
in various school systems in the United States, and a report on 
a conference held by the Board of Examiners, also involving

_SJteacher selection procedures (App. 161-62a).
In addition, Dr. Barrett stated that he had examined all 

the documents relied on by the Examiners' four testing experts 
as the basis for their opinions concerning the Examiners' pro- 
cedures. Dr. Barrett concluded that mere examination of these
documents could not support a claim for content validity (App. 
l62-63a), and that such a claim could be justified only by the 
kind of study which it was clear from their affidavits the 
Examiners' experts did not conduct —  that is, a thorough 
analysis of the various supervisory jobs at issue, establishment 
of success criteria and determination of whether the tests at 
issue adequately represented the most important aspects of the 
related jobs (App. 157-58a) (see also 112-14a). Indeed Dr. 
Barrett noted that the Examiners' experts had not even claimed 
that the examinations were in fact content valid but only that 
it -appeared that the Examiners attempted to design content valid 
examinations (App. 157a).

8 / Each of these documents are described in the appendix to 
Dr. Barrett's affidavit which appears at App. Ex. 15. The 
documents themselves were filed as Exhibits 2-8 to Barrett 
affidavit, Ind. Doc. No. 68.
9_/ (App. 158a) (See supra p. 7 ). These documents were also 
filed as Exhibits to the Barrett affidavit , Ind. Doc. No. 68.



In response the Examiners, who had previously declined 
the court s offer of an opportunity to submit additional evidence 
by affidavit or oral testimony (Ind. Doc. No. 26, TR 105, 109, 110), 
chose to submit only another affidavit from defendant Examiner 
Rockowitz which added nothing to his previous claims (App. 165-77a).

(2) The District Court's Decision
On the basis of a voluminous record, briefly summarized 10/

above, Judge Walter R. Mansfield, who had presided over every 
aspect of the case from its initiation, issued his opinion July 14,
1971 (App. 179-235a), finding that the Board's examinations had the 
"effect of discriminating significantly and substantially against 
qualified black and Puerto Rican applicants." (App. 201a; see also 
190-91a, 221a).

The court's analysis of the comparative pass rates revealed
by the court-ordered examination Survey was based on the affidavits
submitted by statisticians for both sides and the evidentiary

11/hearing held May 21, 1971 (App. 190-201a). The court found that
analysis of the aggregate data for the entire series of examinations 
showed whites passing at almost one and one—half times the rate of 
blacks and Puerto Ricans (App. 190a, 194-96a). The court noted 
fur"ther that whites had passed the examination for Assistant Principal,

_10/ The court noted that in reaching its decision it had had the 
benefit of "a plethora of lengthy affidavits and exhibits, a hearing 
at.which oral testimony was taken, a series of arguments, and extensive 
briefing of the law and facts by the parties," as well as briefs bythree amicus parties (App. 183a).
11/ To the extent that the parties' statisticians differed, the court 
noted that it was more persuaded by the testimony of plaintiffs' 
statistician after "reviewing their testimony and appraising them as witnesses" (App. 194a).

12



Junior High School at almost double the rate of blacks and 
Puerto Ricans and had passed the examination for Assistant 
Principal, Day Elementary School at a rate one-third greater than 
blacks and Puerto Ricans, and that these examinations were of 
particular significance both because of their size and "because 
the assistant principalship has traditionally been the route to and
prerequisite for the most important supervisory position, Principal"

12/(App. 191-92a). Finally, the court concluded, on the basis of 
the expert evidence, that the fact that whites passed at a higher 
rate than blacks and Puerto Ricans in 25 of the 32 examinations
subject to comparison, was proof of the discriminatory impact

13/
of the entire series (App. 193-94a).

However, the court did not rely solely on the statistical 
disparities revealed by the Survey since it was clear from the 
evidence that they represented a gross underestimate of the dis­
criminatory impact of defendants' supervisory examination system.

12/ The court noted that of the 50 examinations covered by the 
survey some were taken by very few people and because of the smallness 
of the samples the results of each, analyzed individually, could not 
be accorded much significance. Thus 41 of the 50 examinations were 
taken by only 83 (or 10.1%) of the total number of black and Puerto 
Rican candidates (App. 196-97a). On all nine examinations taken by 
10 or more black and Puerto Rican candidates, whites passed at a 
substantially higher rate (App. 197a).
13/ The results of Dr. Cohen's statistical analysis are summarized
at App. Ex. 26.

It is worth noting that there was no significant conflict 
between the two statisticians at the evidentiary hearing below.
Dr. Jaspen, the Examiners' statistician, conceded that Dr. Cohen's 
analysis of the aggregate data was correct (Ind. Doc. No. 26,
TR 66-67); that it was proper to subject the aggregate data to

13



Thus the court found that:
"The fact that the process involves a series 
of examinations and that to reach the top 
one must pass several examinations at different 
times in his or her career serves to magnify 
the statistical differences between the white 
and non-white pass-fail rates." (App. 192a)

Thus, for example, if any given examination screened blacks and
Puerto Ricans out at approximately twice the rate of whites, two
successive examinations would screen them out at four times the
rate of whites (App. 192-93a).

On the issue of whether examinations with such an impact could
be justified the court held:

"The Constitution does not require that minority 
group candidates be licensed as supervisors in 
the same proportion as white candidates. The 
goal of the examination procedures should be to 
provide the best qualified supervisors, regard­
less of their race, and if the examinations appear 
reasonably constructive to measure knowledge, 
skills and abilities essential to a particular 
position, they should not be nullified because 
of a de facto discriminatory impact."

[Emphasis added] (App. 201a)
The court concluded, however, that the Examiners' procedures could 
not be justified as being reasonably related to job performance.
On the basis of extensive evidence and exhaustive briefs discussing 
the law and authorities on employment testing,the court found that 
for a test to be considered job-related on the basis of its

13/ (Continued)
analysis to determine the overall impact of the supervisory 
examinations on candidates (id. at TR 68-6 9) ; and that it was 
proper to compare the number of examinations whites passed at a 
higher rate with the number of examinations blacks and Puerto 
Ricans passed at a higher rate, by the "binomial" or "sign" test, 
in assessing the impact of the series of examinations (id. at 
TR 69-71) (apart from the problem of overlap discussed Toy the court 
at App. 195-96a).

14



“.£2nt.?nt validity," it was essential to first perform an 
adequate job analysis:

"Such an analysis requires a study to be made 
of the duties of the job, of the performance 
by those already occupying it, and of the 
elements, aspects and characteristics that 
make for successful performance. Questions 
are then formulated, selective procedures 
established, and criteria prepared for 
examiners that should elicit information 
enabling them to measure these characteristics, 
skills and proficiency in a candidate and 
determine his capacity to do the job satis­factorily. (App. 206a)

The court noted that the Examiners' own expert, Dr. Thorndike, 
had stated that "predictive validity,'1 which depends upon a 
showing of a correlation between test scores and job performance, 
was "most desirable. . . .  to show that the test is in fact 
effective in discriminating between those who are and those who 
are not successful in a particular job," and that anything 
short of that must be recognized as a "stop-gap" (App. 206a).

Although plaintiffs had contended that the Board's examinations 
could not be considered job-related without a showing of 
p redictive; validity, the court did not have to reach this issue.
It found that even accepting the Examiners' position that use 
of procedures designed to ensure content validity was sufficient 
(App. 212-13a), the Board had not "achieved the goals of con­
structing examination procedures that are truly job-related" 
since:

Despite its professed aims the Board has not 
m  practice taken sufficient steps to insure 
that its examinations will be valid as to 
content, much less to predictiveness."

(App. 213a)

15



claim that they relied on outside experts and lay persons in
determining success criteria was not supported by the evidence
(App. 213-14a), that the Chancellor had found the examinations a
barrier to selection of the most qualified supervisors (App. 214-15a),
and that the Examiners' position was not supported by the research
reports they had relied on ". . .as demonstrating the content
validity of . . . supervisory examinations." (App. 215-16a).
The court found that the only report even relating to validity was
a pilot predictive validity study which "showed that there was
little or no correlation between success on the tests and job
success." (App. 216a). The court concluded that the Board had
failed to achieve the goal of content validity, noting that this
conclusion, "which is based upon our appraisal of affidavits of
experts furnished by the parties, is confirmed by our own study

14/
of some of the examinations. . . . "

The court concluded that preliminary relief should be 
granted since there was "a strong likelihood that plaintiffs 
will prevail on the merits at trial,""the balance of hardships 
tips decidedly in favor of plaintiffs, and, pending final determi­
nation of the merits, the effect of preliminary relief would be 
to preserve the status quo until the issues are resolved," and 
preliminary relief would not harm the public (App. 223-25a).

In support of this finding the court noted that the Examiners'

14/ App. 217a. The court noted that defendants' examinations 
typically contained both short-answer, multiple-choice tests and 
essay tests, both of which appeared "aimed at testing the candidate's 
ability to memorize rather than the qualities normally associated 
with a school administrator" (App. 217a), and that a leading 
training manual for such examinations emphasized the use of mnemonic 
devices (App. 217-19a). See 232-25a, n. 23-24 for examples.

16



(3) The Preliminary Injunction
At defendants’ request and over plaintiffs’ vigorous 

objection, the court delayed entry of the preliminary injunction 
because counsel for defendants had alleged they were unable to 
consult with their clients during the summer.^

The Examiners took advantage of this time to develop 
partial statistics and prepare new affidavits relating to the 
November 3, 1970 Elementary School Principals' Examination and 
xn September, 1971, urged the court to consider this new evi­
dence and to permit licenses to be issued and regular appoint­
ments made on the basis of that examination. Despite the fact 
that the motion for a preliminary injunction and the court's 
July 14 opinion were focused largely on that very examination, and 
the record was closed, the court nonetheless carefully considered 
the Examiners' papers, rejecting their relevance for reasons 
set out in detail in the memorandum opinion supporting its 
preliminary injunction (Ag£. 251-55a). After considering a 
variety of proposed injunctive orders submitted by the parties, 
together with supporting affidavits, memoranda and other papers, 
the court entered its preliminary injunction on September 20,
1971 (A££. 2 57-59a), one year after suit was filed, incorporating 
certain of plaintiffs' suggestions, but permitting any party to 
apply for a modification of the order to permit the institution 
of new examination procedures (App. 259a).

.15/ The court meanwhile entered an interim inj 
defendants from conducting further examinations 
eligibility lists, and using outstanding lists 
appointments (App. 237a).

unction restraining 
, publishing new 
to make permanent

17



(4) Subsequent Developments
At the time the court entered its injunction the Chancellor 

had submitted a resolution proposing the institution of an 
interim system for the selection of acting supervisors to 
operate pending final determination of the instant action 
(App. Ex. 27), which the court below noted as bolstering its 
previous determination that the relief endorsed would in no 
way harm the public or the administration of New York City 
schools (App. 255a). A slightly modified version of this pro­
posal was adopted by the Board of Education on October 6, 1971, 
and is presently being implemented. It contains provisions 
for the development of "written procedures governing the 
selection and assignment of acting supervisory personnel,"
"the description . . .  of clear performance objectives for the 
position to be filled," the development of "performance criteria," 
the "periodic evaluation of on-the-job performance of acting 
supervisory personnel," and an "Advisory Council on the 
Selection of Acting Supervisory Personnel" to ensure that 
aPpointments are made in consonance with merit and fitness.
The resolution also established new eligibility requirements 
for acting supervisory positions. 16/

On November 9, 1971, the Examiners submitted to plaintiffs 
a proposal for a new supervisory examination system (attached 
hereto as App. A, pp. 2-13a infra). This proposal was sub­
mitted as the basis for an application to the court for modi­
fication of the preliminary injunction (pp. l-2a, infra).

15/ A  *  •Acting supervisors must meet eligibility requirements for
2 ; ro ^ a ? r ; ? ^ p^ - j r rvisory exa” ination -

18



Designed to improve the "compatibility" of the examinations
with a decentralized system (p. 3a, infra), this proposal
makes clear in describing the steps that would be taken in
developing job analyses, job-related examinations and
validation procedures that previous procedures were inadequate
to ensure job-relatedness. Thus for example, the proposal
states that the Board will obtain

" . . .  in addition to statements of duties of 
the position, appropriate skills, qualities and 
behaviors, in other words, specific job analyses 
for each position for which it is required to con­
duct examinations." (p. 4a, infra)

The proposal describes the manner in which job related examinations 
will be developed, noting that

"[t]he language mastery element in written tests 
will be deemphasized to take its place alongside 
such data as are obtained through other tests 
which reveal essential elements of administrative 
ability, supervisory ability, attitude toward the 
learning process for the varying student popu­
lations of the city, human relations skills as 
the candidate may offer, e.g., knowledge of spoken 
Spanish, knowledge of Afro-American and Hispanic 
culture, etc." (p. 5a, infra)

The proposal suggests that instead of eliminating candidates 
by rigid cut-off scores, as in the past, the Examiners would 
instead provide an "assessment profile" for each candidate, 
thus enabling such community school board to evaluate the 
various candidates in the light of its own particular needs; 
only those candidates who could not satisfy a level of mini­
mum competence would be disqualified by the Board's examinations 
(pp. 6-8a, infra). Finally, the proposal provides for the 
predictive validation of the new examination system (pp.ll-12a, 
infra).

19



Subsequently, the Educational Testing Service of Princeton, 
New Jersey (E.T.S.), submitted to plaintiffs and the Examiners 
a draft proposal for the development by E.T.S. of a truly non- 
discriminatory and job-related examination system. This pro­
posal (attached hereto as App. B, pp. 14-24a,infra) reveals 
beyond any doubt that the Examiners' previous procedures were 
wholly inconsistent with professional test development standards 
and incapable of producing job-related examinations. The pro­
posal also makes clear that a job-related examination system 
can be developed, describing in some detail the steps that 
should be taken to develop job analyses, success criteria, and 
test specifications. Finally, the E.T.S. proposal shows that 
empirical validation, which rests on a demonstrated correlation 
between test scores and job performance, is not only feasible 
but accepted by professionals in test technology as a neces­
sary preliminary to use of an examination (pp. 19a, 22-23a, 
infra). It is also significant that E.T.S. starts with the 
assumption that a job-related and non-discriminatory examination 
would not reject members of different ethnic groups out of 
proportion to their numbers in the applicant group (p. 16a, 
infra).

20



ARGUMENT

Introduction

In reviewing the decision below it is important to realize 
that the trial judge who presided over every phase of the case 
during the entire 12 months of proceedings, acted on the basis 
of an exhaustive factual record, which included literally 
dozens of affidavits, hundreds of pages of exhibits, and an 
evidentiary hearing. The court also relied upon numerous 
briefs, submitted at various stages of the case by the parties 
and three amici curiae, and several oral arguments. Through­
out the proceedings the court proceeded with extreme caution, 
providing only such interim relief as seemed essential, and 
refusing to make any assumptions as to the discriminatory 
impact of defendants’ examination system in the absence of a 
comprehensive summary of examination results. At every stage 
the court offered the parties full opportunity to present 
evidence and argument. While the Examiners now object to 
certain of the court's factual conclusions they could not 
and did not object at any time that the court prevented them 
from presenting their case in whatever form they chose.

It is also important to realize the context in which the 
legal issues involved in this case appear. There has been 
increasing recognition in recent years that New York City's 
schools are in a state of crisis, that ghetto schools in par­
ticular are failing to educate their students, and that black

21



and Puerto Rican children not only do badly as compared to 
whites but do worse the longer they stay in these schools.-^
It is in large part out of this recognition that the movement 
for decentralization grew, in the hope that schools would 
thereby become more responsive to the needs of their com­
munities and more able to serve their particular student 
bodies. It is particularly ironic in this context that 
defendants should maintain an examination system which is 
not job-related and which in no way accounts for the fact 
that different qualifications might be required for positions 
in different schools and communities - a system which locks 
out of the supervisory ranks many persons who are particularly 
qualified to serve the students in the ghetto schools which 
are at present failing their educational task.-̂ "^

17/ „See Note, supra n. 4, at 387-88 and n. 115; Council of 
Supervisory Associations v. Board of Education, 23 N.Y.2d 458, 
463, 297 N.Y.S.2d 547, 551, 245 N.E.2d 204, 207 (1969).
18/ Two recent cases have upheld administrative decisions to 
by-pass normal civil service lists for school supervisory 
positions, largely in recognition of the fact that traditional 
civil service procedures have failed to select the very people 
most needed to relate to ghetto communities and thereby serve 
the schools which are most in need of improvement. In Council 
of Supervisory Associations v. Board of Education, 23 N.Y.2d 
458, 297 N.Y.S.2d 547, 245 N.E.2d 204 (1969), the New York 
Court of Appeals upheld the Board of Education's decision to 
create the new position of Demonstration Elementary School 
Principal, and to make acting appointments to said position 
pending development of a relevant examination, thereby by­
passing the Examiners' Elementary School Principal list.
The court's decision was premised in part on its recognition, 
in light of New York's educational crisis, of the importance 
of selecting principals who could relate to their communities 
and its finding that the traditional examination system had 
failed to select such principals. See also Porcelli v. Titus,
302 F. Supp. 726 (D. N.J. 1969), aff'd 431 F. 2d 1254 (3d Cir.. 1970). -----

22



The evidence below revealed the gross disparity between 
the numbers of blacks and Puerto Ricans on the supervisory 
staff and in the student body (Ap£. 187-88a). Thus blacks 
and Puerto Ricans constituted over half of the student body 
but only one per cent of the principals. Whatever factors 
may have contributed to this in the past it is clear that, with 
decentralization and with revised eligibility requirements, 
the Examiners' procedures had become the ma]or barrier to the 
employment of minority supervisors.— ^  The evidence below

12/  Even if the examinations themselves did not disqualify 
blacks and Puerto Ricans at such significantly disproportionate 
rates, they would nonetheless, given the exclusionary effect 
of the system as a whole, constitute unconstitutional barriers 
to defendants' affirmative obligation to license and appoint 
qualified black and Puerto Rican supervisors. The Survey 
illustrated the manner in which defendants' system operated 
as a whole to effectively prevent the appointment of any sig­
nificant numbers of blacks and Puerto Ricans to the super­
visory ranks, because of the combination of such factors as 
low pass rates, eligibility requirements and exhaustion of 
lists. See Plaintiffs' Memorandum on Relevance of Statistical 
Tabulations, Ind. Doc. No. 60, pp. 10-12. A system of selection 
which operates to perpetuate the gross underrepresentation of 
blacks and Puerto Ricans on that staff and to prevent the 
appointment of blacks and Puerto Ricans who might be qualified, 
violates the Equal Protection Clause of the Federal Constitution. 
See generally Porcelli v. Titus. 431 F.2d 1254, 1257-58 (3d 
Cxr. 1970), affirming 302 F. Supp. 726 (D. N.J. 1969), holding 
that where there was a gross disparity between the percen­
tage of blacks in the student population and their percen­
tage on the supervisory staff, the school board had an affir­
mative obligation under the Federal Constitution to correct 
that disparity. See also Jackson v. Wheatley School District.
430 F.2d 1359, 63 (8th Cir. 1970); Armstead v. Starkville 
Municipal Separate School District. 325 F. Supp. 560, 569 
(N.D. Miss. 1971), where the courts found disparities between 
the percentage of black students and the percentage of black 
faculty members evidence of racial discrimination. In North 
Carolina Board of Education v. Swann, 402 U.S. 43 (1971), the 
Supreme Court made it clear that any law, regulation or of­
ficial policy which prevented boards of education from taking

23



further revealed that the Examiners' procedures were blocking 
the appointment of minority and other candidates not on the 
basis of "merit and fitness" but, rather, on the basis of 
their inability to answer such questions as "[W]ho killed 
Cock Robin?" (App. 232a, n. 23).

The Board's procedures have for decades been subjected 
to severe criticism by every body which has investigated it 
because, far from fulfilling their purported goal of selecting 
the best qualified supervisors, they were in fact locking such 
persons out and perpetuating a sterile, bureaucratic, insider 
system.20/ But despite this criticism and series of recom­
mendations for reform, no significant change in the Board's 
procedures took place. It is only as a result of the decision

19/ (Cont'd)

affirmative action to correct the kind of gross racial under­
representation exhibited in the instant case must be declared 
unconstitutional. See generally Fiss, A Theory of Fair Employ­
ment Laws, 38 U. CHI. L. REV. 235, 270 n. 42 (1971).

20/ See Mayor's Committee on Management Survey, Administrative 
Management of the School System of New York City (1951) (the 
"Strayer and Yavner Report"); Schinnerer, A Report to the New 
York City Education Department (New York, 1961); Cresap,
McCormick and Paget, Summary Report of Assignments Conducted 
for the New York City Board of Education (1962); GRIFFITH REPORT, 
TEACHER MOBILITY IN NEW YORK CITY: A STUDY OF RECRUITMENT, 
SELECTION, APPOINTMENT, AND PROMOTION OF TEACHERS IN THE NEW 
YORK CITY PUBLIC SCHOOLS 162-230 (1963); Center for Field 
Research and School Services, New York University, A Report 
of Recommendations in the Recruitment, Selection, Appointment 
and Promotion of Teachers in the New York City Public Schools 
(1966); BUNDY REPORT, MAYOR'S ADVISORY PANEL ON DECENTRALI­
ZATION OF THE NEW YORK CITY SCHOOLS, RECONNECTION FOR LEARNING 
110, n. 28 (1967); Rogers, 110 LIVINGSTON STREET 285-97 (1968).
In addition, Boards of Education have, in recent years, urged 
the legislature to abolish the Board of Examiners. And most 
recently the New York City Commission on Human Rights issued a

24



of the court below that there now appears to be some hope for 
the development of a selection system which will truly allow 
for the appointment of those supervisors best qualified to 
serve the schools and children of this City (see pp. 18-20, supra).

POINT I
THE DISTRICT COURT'S FINDING THAT 
DEFENDANTS' EXAMINATION SYSTEM HAD 
A SIGNIFICANT AND SUBSTANTIAL DIS­
CRIMINATORY IMPACT WAS CLEARLY 
SUPPORTED BY THE RECORD

Throughout their brief the Examiners claim that the court 
below based its decision on statistical rather than legal 
analysis, and found a prima facie case of discrimination made 
out on the basis of findings of mere "statistical significance" 
and departures from "perfect equality" in the treatment of

20/ (Cont'd)
thorough report on the schools' selection system, based on 
public hearings held Jan. 25-29, 1971 and extensive documen­
tation, recommending that the Board of Examiners be dis­
continued on the grounds that its present examination system 
locked out qualified minority and other candidates, and was 
apparently not job-related. The Commission recommended further 
that New York City rely on state certification for initial 
screening (as does virtually every city school district in 
the country) and on community school boards for actual 
selection based on "criteria and procedures geared to the 
need of individual boards." Equal Employment Opportunity and 
the New York City Public Schools, An Analysis and Recommen­
dations Based on Public Hearings Held January 25-29, 1971 by 
the City Commission on Human Rights, (pp. xv-xvi). The Com­
mission specifically found that the lists of job duties used 
to prepare examinations were "inadequate bases for sound 
test construction" (p. xx) and that the Board of Examiners’ 
resources were inadequate to the task of designing and adminis­
tering tests and assessing their validity (p. xxi).

25



members of different ethnic groups. (See Brief pp. 10, 15-17, 
21-24) This is simply not true. The court's decision was 
clearly based not on a test of mere statistical significance 
but, rather, on findings that defendants' examinations had a 
significant and substantial discriminatory impact, resulting 
in "gross" differences in the treatment of different ethnic 
groups (App. 201a, 191a, 221a; see also 190a).

The court's finding of discriminatory impact was based 
essentially on two factors. The court relied first on the 
results of the survey, which showed that the entire series 
of examinations had a significant and substantial discrimi­
natory impact on black and Puerto Rican applicants and also 
showed gross disparities in pass rates on the biggest and 
most important examinations (pp. 12-13, supra; App. 189-94a* 
App. Ex. 26). But secondly, the court concluded that the 
disparities revealed by the survey constituted a gross under­
estimate of the discriminatory impact of the examination 
system because, as the evidence revealed and the court found, 
during the period covered by the survey the promotional process 
involved a series of examinations and "to reach the top one 
must pass several examinations" (App. 192a). The court found

21/ Indeed, the reason the court apparently felt called upon 
to discuss the statistical analysis of the data in such detail 
is because the Examiners had rested their case largely on an 
attempt to attack the statistical significance of the survey 
results. See Dr. Jaspen's affidavit, Ind. Doc. No. 66 and 
May 21 hearing TR, Ind. Doc. No. 26.

26



that the net effect of this system was to screen minority 
candidates out at rates far in excess of those indicated by 
the survey results (App. 192-93a). ii/

22/
The Examiners miss the point in their brief (pp. 18-19) 

since they assume erroneously that the example the court gives 
to illustrate the manner in which this multiplier effect 
worked was intended as some kind of factual future prediction.

The Examiners introduce in their brief at p. 19 partial 
and misleading pass-fail data on the results of the November 
1970 Elementary School Principals' Examination. These data, 
produced after the court's decision, when plaintiffs had no* 
opportunity to present evidence, were rejected below in the 
court s memorandum opinion supporting the preliminary injunction 
(Ap£. 251-55a), on the grounds that they were too incomplete 
to be meaningful (covering only 60% of the candidates), that 
they revealed that the pass rates for the identified candidates 
could not be assumed to be typical of the pass rates for those 
not identified, that plaintiffs had had no opportunity for 
cross-examination, and that plaintiffs were entitled to 
discovery to determine all relevant facts relating to this examination (App. 253-54a).

The fact that the Board of Education has recently re­
vised certain eligibility requirements is irrelevant since 
during the period of the survey the previous requirements 
were in effect. Moreover, despite the revisions most candi­
dates must still pass through a succession of tests starting 
with the teacher's examination. Finally and most important, 
to the extent that the Board of Education does reduce the 
number of examinations through which candidates must pass as 
they climb the promotional ladder, it is likely that the dis­
criminatory impact of each particular examination (assuming 
that their nature remains unchanged) will increase. This is 
because candidates who have survived screening by prior tests 
are relatively homogeneous in test-taking ability and un­
screened groups are likely to show greater differences between 
different ethnic groups (Dr. Cohen's affidavit, ver. 5/6/71,Ind. Doc. No. 62, para. 7).

It is worth noting in this connection that the survey 
results grossly underestimate the discriminatory impact of 
defendants examination system in an even more significant 
way, not taken into account by the court below. Virtually 
all candidates covered by the survey had been pre-screened

27



In addition, the court relied on the fact that cities 
which did not use an examination system like New York1s had much 
higher percentages of blacks and Puerto Ricans on their super­
visory staffs (App. 197-98a; 222a; see also p. 3. supra)

The Examiners also quarrel with the court's analysis of 
the survey results in numerous respects. In considering their 
arguments it is important to realize that the survey procedure 
was developed by the parties and the court as the best method 
of assessing the impact of defendants' examinations; that the 
parties' statisticians differed on no significant issues (see 
n. 13 , supra); that defendants had ample opportunity to present 
any evidence they considered relevant below; and that the
court made its findings on the basis of exhaustive evidence 
and briefing.

22/ (Cont'd)

by the Board's teacher examinations and it is likely that it 
was at this stage, where the applicant group was unscreened 
by any previous test, that the discriminatory impact was 
greatest. what is extraordinary is that the Board has managed 
to design examinations which discriminate even against blacks 
and Puerto Ricans who have proven themselves to be good test
t.c l.K 6  ITS •

23/
, fvTh! Examiners claim that these disparities are explained 
bT th^-faCt that New York city's education and experience eligibiiity requirements were higher than those in "many other cities (Brief pp. 20-21), but there is no evidence in the 
record that this is true with respect to any specific city named by the court. J

28



The Examiners contend now, for the first time, that the 
comparative pass rates revealed by the survey are irrelevant, 
on the grounds that the minority applicant group allegedly 
constitutes a disproportionately large percentage of the pool 
of eligibles and is allegedly less qualified than the white 
applicant group (Brief pp. 11-15). This argument depends on 
a number of wholly unsupported and unwarranted assumptions: 
first, that the teacher population approximates the pool of 
eligibles; second, that the ethnic groups within this popu­
lation are equivalent in ability; and third, that among those 
®^icfihle it is the most qualified who apply for promotional 
examinations.

Presentation of this issue now is, however, untimely 
since it was not raised below where there would have been 
ample opportunity to develop and present relevant evidence. 
Instead the Examiners insisted below, and the court conceded, 
that the best evidence of the examinations' discriminatory 
impact would be comparative pass rates of actual applicants. 
(PP-4, 9, supra) Thus the Examiners took the position below 
that "the only meaningful statistic would be a comparison of 
the pass-fail ratio of black and Puerto Rican applicants"
(ftPP- 88a). At no point in the proceedings below did they 
indicate that these were not the most relevant data. Before 
the survey their claim was that blacks and Puerto Ricans 
passed at the same rate as whites. After the survey their 
claim was that there was no significant difference in the 
pass rates revealed. it is only now that the court below has

29



found that the differences are in fact significant and sub­
stantial that the Examiners claim that the entire survey and 
all the proceedings below related to it were an exercise in 
futility because the focus should have been on the pool of 
eligibles rather than the applicants. a e  Examiners can_

not now attempt to justify the disparities in pass rates 
revealed by the survey on the basis of assumptions - for 
which there is no support in the record - that the minority 
applicants were less qualified than the white applicants.^ 

Moreover, there is no reason to credit the Examiners' 
unsupported assumptions that the minority applicants were less 
qualified. First there is no reason to assume that the teacher

24/
bSl?W P o n t i f f s  attempted to obtain statis-ics on the pool of eligibles defendants refused to provide them. See p. 9 , supra. ^

25/
*£deed below there was uncontradicted expert testimony 
he analysis of the survey results was relevant "for the 

population at issue, not merely the sample under study." 
(Hearing May 21, Ind. Doc. No. 26, TR 13).

30



population approximates the pool of eligibles.^ Secondly 
different ethnic groups in the teacher population are not 
necessarily equally qualified. Since only highly motivated 
minority persons are likely to try to break into a system 
that has traditionally been closed to them, the minority group 
may be particularly well-qualified. Finally, there is no 
reason to assume that in this school system promotional 
examinations will attract the most qualified candidates 
from every ethnic group. The most talented teachers may well 
decide to leave the school system after a few years. This 
would likely be particularly true of whites who have more 
opportunities open to them. The less talented and more 
system-oriented whites, on the other hand, might well tend to 
seek supervisory positions. Disproportionate numbers of highly 
qualified blacks and Puerto Ricans might, however, 
apply for promotional examinations because it is only in recent

26/
Substitute teaching and acting supervisory experience is 

often an alternative to licensed experience and blacks and 
Puerto Ricans represent a much larger percentage of substitute 
teachers and acting supervisors. Similarly, experience as a 
teacher anywhere in the United States under a valid license 
satisfies the requirement of previous teaching experience.
And since the qualifications for each position are different, 
there is simply no way of estimating the pool of eligibles 
for the various examinations. See generally Exhibits 1-9 of

ver- 1-0/19/71. fid. DOC. No. 44. for examplesof qualifications. r

31



years that opportunities for appointment have begun to open 
up and, therefore, that these examinations represent a real 
opportunity for advancement. By contrast, the white applicant 
group more likely includes large numbers of persons who were 

previously and failed previous examinations.
While one could speculate endlessly as to the comparative 

qualifications of the different ethnic applicant groups the 
point is that the evidence below showed that the examinations 
had a substantial discriminatory impact on black and Puerto 
Rican applicants. The Examiners failed to provide any evi­
dence justifying that impact, although they had ample oppor­
tunity to document their hypothesis that the black and Puerto 
Rican applicant group was less qualified than the white. in 
the absence of evidence, such an hypothesis cannot be accepted 
by this court, particularly since there is every reason to 
believe that the black and Puerto Rican applicants constituted 
a highly motivated and qualified group who persisted within a 
system traditionally closed to them.

The Examiners' other objections to the court's analysis, 
also raised here for the first time, are equally frivolous.
Thus they claim that the survey results cannot be used to 
assess the discriminatory impact of defendants' examinations 
in general, because examinations for different supervisory 
positions were involved (Brief pp. 17-18). However, this 
issue was-thoroughly explored at the evidentiary hearing 
below and the court concluded, on the basis of testimony 
by both statisticians, who evidenced no disagreement on this

32



issue, that valid conclusions as to the discriminatory nature 
°f the examinations could be drawn even though discrete 
examinations for different positions were involved. More­
over the purpose of the survey designed by the parties and the 
court was to assess the impact of the examinations as a whole , 
and the Examiners' only argument at that stage of the pro­
ceedings was that fewer examinations should be included. The 
Examiners also claim that the court erred in relying on com­
parative pass rates for the entire group of applicants. They 
contend that it should have considered as failing only those 
who completed the entire examination process and not those 
who failed because they withdrew at some earlier stage, either 
before or after the written examination. But the court's use 
of comparative pass rates was clearly reasonable and supported 
by the expert evidence below. Moreover, both statisticians 
testified that it made no difference whether the survey results 
were analyzed by including or excluding the group of candidates

iZ/ See Hearing May 21, 1971, Ind. Doc. No. 26. Thus Dr. Cohentestified:
Q. Is analysis of aggregate test results relevant 

to the sort of questions we have discussed when 
there are different tests involved for different 
kinds of supervisory positions and when the tests 
involve different numbers of candidates?

A. Yes, they are quite adequate. (TR 9-10)
Q. Is it correct to use the binomial test where 

you have a series of different examinations 
for different positions?

A. Absolutely. . . . (TR 18)
See generally Dr. Cohen's testimony at TR 9-10, 13,14,18, 19-20,
22 and Dr. Jaspen's testimony at TR 68,70,71,81-82, 89.

In addition testimony by both statisticians completely ex­
ploded the Examiners' argument below, repeated in its brief at 
p. 17, that there was any relevance to the failure to find 
statistically significant differences in pass rates in various 
examinations with extremely small samples. See generally TR 22-32, 71-78.

33



who failed to complete the examination process.— '

POINT II
THE COURT BELOW CORRECTLY RULED THAT A PUBLIC 
EMPLOYER VIOLATES THE EQUAL PROTECTION CLAUSE IN 
CONDUCTING A PROMOTIONAL EXAMINATION SYSTEM WHICH 
SYSTEMATICALLY AND SIGNIFICANTLY DISCRIMINATES 
AGAINST ETHNIC MINORITY GROUPS WHERE THAT EXAMI­
NATION SYSTEM CANNOT BE SHOWN TO HAVE ANY RELATION TO JOB PERFORMANCE

A ' A Prima Facie Case of Discrimination is Made Out
Where—Examinations Significantly and Systematically 
Exclude Members of an Ethnic Minority Group 
Regardless of Whether Any Subjective Intent to 
Discriminate is ShownT
The E <aminers argue in their brief at pp. 21-26 that even 

where there is evidence of grossly disparate treatment of racial 
groups, additional evidence of purposeful discrimination is 
essential to establish a claim under the Equal Protection Clause.

The danger of their argument, if accepted, is that it would 
leave courts powerless to deal with the kinds of discrimination
problems that ethnic minorities will face increasingly as overt29/
forms of bias are outlawed. Moreover the interests of those

28/ see Hearing May 21, Ind. Doc. No. 26, TR pp. 10-11, 55. Thus 
Dr. Jaspen, defendants' statistician, testified:

We analyzed this with pass versus fail plus did 
not appear, plus withdrawn, and also just pass 
versus fail, and we obtained completely consistent 
results. . . . (TR 55)

.29/ Thus an Griggs v . Duke Power Co.. 420 F.2d 1225 (4th Cir. 1970), 
rev_d, 401 U.S. 424 (1971), Judge Sobeloff noted in dissent:

The case presents the broad question of the use of 
allegedly objective employment criteria resulting in the 
denial to Negroes of jobs for which they are potentially 
qualified. . . .  On this issue hangs the vitality of 
the employment provisions (Title VII) of the 1964 Civil 
Rights Act: whether the Act shall remain a potent tool
for equalization of employment opportunity or shall be 
reduced to mellifluous but hollow rhetoric.

34



discriminated against are essentially the same and deserve the 
same degree of protection whether employment opportunities are 
denied explicitly and intentionally or inadvertently. it is
for such reasons that this and other courts have made it clear that 
where there is evidence of a racial impact or classification, no

30
evidence of an overt or covert intent to discriminate is necessary — ' * S

29/ (Continued)

The pattern of racial discrimination in employment 
parallels that which we have witnessed in other areas. 
Overt bias, when prohibited, has ofttimes been supplanted 
by more cunning devices designed to impart the appearance 
of neutrality, but to operate with the same invidious 
effect as before. . . . (420 F.2d at 1237-38)

See also the Supreme Court’s opinion, 401 U.S. at 431.

S®e ' 2̂3.*' Norwalk CORE v. Norwalk Redevelopment Authority.
395 F.2d 920, 931 (2d Cir. 1968) (finding irrelevant the fact that 
discriminatory effect of an urban renewal housing plan is "accidental " 
since Equal Protection Clause prohibits thoughtlessness and not -just 
governmental design to discriminate, citing Hobson v. Hansen, infra)-
T~,n.nedT^^vrk Homgs Ass’n- v. Lackawanna, 436 F.2d 108, 114 (2d Cir 1970) (holding with regard to the City's provision of facilities 
that even if the discriminatory effect resulted from "thoughtlessness 
rather than a purposeful scheme, the City may not escape responsibility 
for placing its black citizens under a severe disadvantage which it 
cannot justify"; Southern Alameda Spanish Speaking Org. v. Union 
Ci^, 424 F . 2d 291, 295 (9th Cir. 1970) (if result of zoning bv 
referendum is discriminatory a substantial Constitutional question 
^presented without regard to racial motive); Hobson v. Hansen.
Ini p •,?"??: j01' 497‘ <D -D -C - 1967). aff'd sub nom Snmck v. Hobson.408 F.2d 175 (D.c. Cir. 1969) {". . . the arbitrary quality of 
thoughtlessness can be as disastrous and unfair to private rights and 
the public interest as the perversity of a willful scheme"; Hawkins
v^ Town of Shaw, 437 F.2d 1286, 1291-92 (5th Cir. 1971) (pendTHS----
decision after rehearing en banc) (finding a prima facie case of 
discrimination made out on the basis of statistical disparities in 
the provision of municipal services to different racial groups although 
the record showed no "bad faith, ill will or . . . evil motive")- 
Cgrmical v. Cravep,9th Cir. No. 26, 236, Nov. 4, 1971 (holding that 
where test disqualified disproportionate number of veniremen 
from jury service, the fact that the system was not designed with 
discriminatory intent was irrelevant); Jackson v. Godwin, 400 F.2d 
529 (5th Cir. 1968). Cf. Powell v. Power. 436 F.2d 84, 88 n. 7 
(2d Cir. 1970). See also, Gaston County v. United States, 395 U.S. 285 
(1969); Smith v. Texas, 311 U.S. 128, 132 (1940) (racial discrimination 
in jury selection unconstitutional "whether accomplished ingeniously or ingenuously").

35



And virtual^ every decided case involving the use of tests 
by public employers has held that where a test has a substantially 
different impact on different racial groups and cannot be justi­
fied as job-related, its use violates the Equal Protection Clause, 
whether or not an intent to discriminate can be found. ^

31/
i .Thus m  Carter v. Gallagher, 3 CCH E.P.D. f8335 (8th Cir.

' M f>™ing m  pertinent part, 3 CCH E.P.D. f8205 (D. Minn. 
1971), panel opinion adopted in pertinent part en banc, No. 71-1181 
8th Cir. Jan. 7, 1972, slip op. p. 11, involving fire depart­
ment hiring practices, the court noted that there had been "no 
express finding of bad faith or evil motives," and held that 
under 42 U.S.C. §§1981 and 1983 there was no "requirement that 
discrimination be willful or intentional" (p. 6993-203) in 
Penn v. Stumpf, 303 F. Supp. 1238, 1243-44 (N.D. Cal. 1970), the 
court found that while there was "no question here of overt 
racial classification," a selection procedure "may appear on 
its face to be fair and neutral, but if in its application a 
discriminatory result ensues, the procedure may be constitutionally 
impermissible" (police dept, tests). Arrington v. Mass. Bav 
iransp. Authority, 306 F. Supp. 1355, 1358 (D. Mass 1969), 
held irrelevant the fact that there was "no contention of any 
intent to discriminate against black or Spanish-speaking persons 
by means of this test," on the grounds that:

. . . It is not enough that the factors producing the
classification and the consequent inequality are them­
selves objectively neutral and without a background of 
even latent discriminatory purpose: when the effect
is to deprive some citizens of rights that should be 
equally available to all, then there must be a compel­
ling justification. . . . "

In Western Addition Community Organization v. Alioto, 330 F.
Supp. 536, 539-40 and n. 4 (N.D. Cal. 1971), the court held'that:

. . . where the hiring practice of a public agency (even
though it does not intend to discriminate against 
minority groups) has the effect of producing a de facto 
pattern of racial discrimination, such a discriminatory 
effect . . . renders the method of selection sufficiently 
suspect to make a prima facie case of unconstitutionality."

(330 F. Supp. at 539)
Ca-stro v. Beecher. 4 CCH E.P.D. f~569, clarified at f7570, judgment 
modified at [̂7589 (D. Mass. 1971), struck down police department 
employment tests on the ground that they had the consequence of 
disqualifying minority applicants at a substantially higher rate 
than wnite applicants and were not job-related, finding specific-

that 'plaintiffs need neither allege nor prove that defendants

36



Supreme Court has made it similarly clear that under Title VII of
the 1964 Civil Rights Act (42 U.S.C. § 2000-e), which outlawed
discrimination by private employers,

"good intent or absence of discriminatory intent does not 
redeem employment procedures or testing mechanisms that 
operate as 'built-in headwinds' for minority groups and 
are unrelated to measuring job capability."

32/Griggs v. Duke Power Co., 401 U.S. 424, 432 (1971).
The Griggs decision, as well as other testing cases decided 

under Title VII, is directly applicable here, since the prohibitions 
^9 îri£3t employment discrimination embodied in the Act, governing 
private employers, coincide with those embodied in the Fourteenth 
Amendment, governing public employers. It would be anomalous in­
deed if public employers had a lesser obligation than private to

32a/afford fair treatment to all ethnic groups.

31/ (Cont'd)
acted with intent or motive to discriminate," citing Griqgs. infra 
(4 CCH E.P.D. f7569 at p. 5276 n. 4; pp. 5282-83). Both Armstead 
v. Starkville Mun. Sep. School Dist., 325 F. Supp. 560, 570 (N.D. 
Miss. 1971), and Baker v. Columbus Mun. Sep. School Dist.. 329 
F^ Supp. 706, 720—21 (N.D. Miss. 1971) held, wholly independent 
of any findings of intentional discrimination, that use of the 
Graduate Records Examination (Armstead) and N.T.E. (Baker) to select 
teachers violated Equal Protection because they disqualified blacks 
at disproportionate rates and were not job-related.
32/ See also, e.g., Johnson v. Pike Corp., 4 CCH E.P.D. f7517 at pp. 
5048, 5050 (Cent. D. Cal. 1971), noting that Griggs was not limited 
to situation of previous overt discrimination and holding unlawful 
a policy of firing persons whose wages were garnisheed even though 
the discrimination effected by such policy was "not really intended 
by the employer;" Robinson v. Lorillard Corp., 444 F.2d 791, 796-97 
(4th cir. 1971); Hicks v. Crown Zellerbach Corp., 319 F. Supp 314 
320 (E.D. La. 1970^ " ~ ~ ---
3_2a/ Thus Fourteenth Amendment prohibitions against discrimination 
kave keen found "at least as great as those levied upon private 
employers in Section 703" of the 1964 Civil Rights Act, Baker, 
s.yj?rg r*- 31. citing and relying on Griggs and other Title VII cases, 
329 F. Supp. at 721. The Act is if anything narrower in its 
coverage since it expressly permits the use of tests provided they 
are not "designed, intended or used to discriminate...." 42 U S C § 2000e-2 (h) .

37



B. Where Employment Tests are Shown to have a Significant and 
Substantial Discriminatory Impact on Ethnic Minority Groups 
They Violate the Equal Protection Clause if They Cannot be 
Shown to be Job-Related.

(1) The Court applied the proper standard in determining that 
an adequate case of discriminatory impact had been made out.

Defendants contend that the court below applied a standard of 
statistical significance, demanding perfect equality in treatment 
of the races, and that this standard is inconsistent with the law. 
(Brief pp. 21-26). As noted supra pp. 26-28, the court did not 
in fact apply such a standard but relied instead on its findings of 
substantial" and "gross" disparities based on the survey results 

and on the multiplier effect of the requirement that candidates pass 
through a succession of tests. This standard was completely
consistent with decided cases in the employment discrimination area, 
which have applied the principle that where an employment policy has 
a differential impact on different racial groups; and cannot be 
justified on non-racial grounds, a case of discrimination is made out.

The instant case is unique not because a particularly strict 
standard was applied but, rather, because the discriminatory impact 
of defendants' examinations was so thoroughly documented. Most 
employment discrimination cases, including the Supreme Court's decision 
in Griggs v. Duke Power, supra, have struck down tests without any 
comparable showing of discriminatory impact. Statistics on comparative 
pass rates of different ethnic groups have usually been either wholly 
unavailable or else hopelessly inadequate as the basis for drawing 
valid conclusions as to the overall impact of the tests. The courts 
have therefore tended to rely on assumptions (based on such factors 
as the nature of the tests, the different cultural and educational 
backgrounds of different racial groups, and low percentages of 
minority persons in the work force) that there is some significant

38



discrepancy between pass rates of different groups. Thus in 
Griggs the Supreme Court struck down tests which the record re­
veals only three candidates took -- two black and one white --

33 /all of whom failed.—  The Court simply had to assume that
the tests operated "to disqualify Negroes at a substantially 
higher rate than white applicants" 401 U.S. at 426 (emphasis 
added), on the basis of findings as to how blacks and Puerto 
Ricans had fared on such tests elsewhere (id. at 430, n. 6).
The Court concluded that they were discriminatory since 
"[b]asic intelligence must have the means of articulation to 
manifest itself fairly in a testing process" (id. at 430). In 
Penn v. Stumpf, supra n. 31 , police promotional tests were 
found discriminatory on the basis of disparities between 
minority representation in the Oakland City population and 
on the Police Department, and an allegation that significant
numbers of blacks had applied. In Carter v. Gallagher, supra

in an opinion subsequently adopted en banc, n. 31 , the Eighth Circuit found/Fire Department examinations
discriminatory on the basis of a showing that the Department 
was all-white, whereas about 6% of Minneapolis' population 
was non-white, and that the test used a formal English vocabu­
lary and would thus tend to eliminate blacks at a higher rate

see Brief for Petitioners, p. 6 and n. 3, and see record 
filed in the U. S. Supreme Court.

39



than whites.34y In Castro v. Beecher, supra n. 31 , the court
concluded that police examinations were discriminatory against 
all "minorities which did not share the prevailing white 
culture, simply because of their verbal, academic and multiple 
choice nature.— / jn Western Addition Community Organization v 
Alioto, supra n. 31, the court relied on disparities between 
minority representation on the Fire Department and in the 
area's general population, ignoring statistics which showed 
that of the two ethnic groups allegedly discriminated against, 
one, the Mexican-American, had passed at virtually the same 
rate as the white group (34% as compared to 36%). * 3 4

34y
3 CCH E.P.D. f8338 at 6993-203; 3 CCH E.P.D. *8205 at 

p. 6669. While some comparative pass rate statistics were 
available, the district court apparently felt it could not 
rely solely on these because the sample of identifiable 
minority candidates was so small. 3 CCH E.P.D. f8205 at 
p. 6669.
35/ 4 CCH E.P.D. ^[5283; see also p. 5284. The court noted 
in addition that black and Spanish-surnamed candidates had 
done less well on the examinations than white candidates, but 
this was not central to its finding of discrimination. 4 CCH
E. P.D. f7659 at p. 5282.

330 F. Supp. at 538. In Gregory v, Litton Systems. 316
F. Supp. 401, 463 (C.D. Cal. 1970), the court struck down an 
employer's policy of disqualifying applicants who had been 
arrested a number of times without any evidence as to the 
impact of this policy on actual applicants. The court simply 
assumed that the policy denied black applicants "equal oppor­
tunity for employment," and disqualified a "substantial and 
disproportionately large number of Negroes," on the basis of 
national arrest figures showing that Negroes had a higher 
arrest rate than whites. In Johnson v. Pike Corp., 4 CCH 
E.P.D. f7517 at p. 5049 (C.D. Cal. 1971), the court found a 
violation of Title VII in a rule authorizing discharge of 
employees whose wages had been garnisheed on the grounds that 
"available information" indicated that "the proportion of 
racial minorities among the group of people who have had their 
wages garnisheed is significantly higher than the proportion of racial minorities in the general population."

40



The few employment cases which discuss statistical dis­
parities between the pass rates of different racial groups 
support a finding of discriminatory impact in the instant 
c a s e . ^  And cases in such areas as jury discrimination 
and reapportionment 39/ have established legal standards 
which compel a finding that the statistical disparities shown 
here make out a prima facie case of discrimination.

See, e.g. , Carter v. Gallagher, supra nn. 31,34, where the district 
court found minority groups apparently did "substantially less 
well" on the basis of partial statistics indicating that 27% 
of identified minority group members passed as against an 
overall pass rate varying between 40% and 65%. 3 CCH E.P.D.
f8205 at 6669.

See, e.g., Turner v. Fouche, 396 U.S. 346, 360 (1970);
Whitus v. Georgia, 385 U.S. 545, 552, n. 2 (1967); Jones v.
Georgia, 389 U.S. 24 (1967); Sims v. Georgia, 389 U.S. 404,
407 (1967). In U.S. v. Criminal Courts of New York, 442 
F.2d 611 (2d Cir. 1971), in refusing to overturn criminal 
contempt convictions on the grounds that the grand jury 
before which the contempts had occurred was unrepresentative, 
this court made it clear that it was dealing with the right 
of the defendants to a fair tribunal and not the rights of 
minority group members to serve as jurors. The primary issue 
was, therefore, whether a case of bias had been made out and 
the court held it had not since there was no apparent ground 
for the "assumption that grand jurors predominantly of one 
race or those over 35, cannot provide a fair hearing for per­
sons of another race or generation" (442 F.2d at 616). And 
in considering New York's system of jury selection the court 
again noted that "the only plausible basis for Chestnut's 
equal protection claim is the possibility that a grand jury 
deficient in members of his race would deal with him dif­
ferently. . . . "  (442 F.2d at 618, n. 12) It relied also on 
the fact that there was no denial "that the racial imbalance 
of the [jury] list was not the result of 'an exclusionary 
device . . .' but was due to New York County's system of
voluntary grand jury service" (442 F.2d at 617-18), a factor 
distinguishing it from the jury cases cited supra. Indeed 
the court specifically found that the under-representation 
of blacks "resulted solely from their relative lower propen­
sity to volunteer" (442 F.2d at 618, n. 12).
-23*1 See, e.g. , Kirkpatrick v. Preisler, 394 U.S. 526, 530-31 
(1969); Wells v. Rockefeller, 394 U.S. 542 (1969).

41



The court's standard is not only consistent with decided 
authority, but seems the only reasonable standard in the em­
ployment testing area. In recent years a number of studies 
have documented the fact that most tests discriminate unfairly 
against blacks and Puerto Ricans because they are designed
for persons with different cultural and educational back-

40/grounds. Recent studies have also documented the fact
that there is generally very little relation between employ­
ment tests and 30b performance,^-^ particularly when such

42 /tests are given to mixed racial groups.— ' And finally.

40/ See, e.g., Kirkpatrick, et al.. Testing and Fair Employ­
ment 5 (1968); J. Coleman, Equality and Educational Opportunity 
219-20 (1966); authorities collected in Cooper and Sobol, 
Seniority and Testing under Fair Employment Laws, 82 HARV. L. 
REV. 1598, 1639-41 pp. 11,13,14,15,16, 17 (1969); Hobson v. 
Hansen, 269 F. Supp. 401, 484-85 (D.D.C. 1967), aff'd sub nom. 
Smuck v. Hobson. 408 F.2d 175 (D.C. Cir. 1969). See also 
Barrett affidavit, App. 36-38a.
41/ See, e.g., E. Ghiselli, The validity of Occupational Apti- 
titude Tests 51, 57 (1966). In Griggs the Supreme Court notedthat:

"The facts of this case demonstrate the inadequacy 
of broad and general testing devices as well as the 
infirmity of using diplomas or degrees as fixed 
measures of capability. History is filled with examples 
of men and women who rendered highly effective perform­
ance without the conventional badges of accomplishment 
in terms of certificates, diplomas, or degrees."

(401 U.S. at 433)
42/ See generally authorities cited in Cooper and Sobol, 
supra n. 40at 1643-46 (1969); Hobson v. Hansen, supra n. 40, 
269 F. Supp. at 484-85.

42



workable standards have been developed by administrative 
agencies charged with enforcing laws against discrimination, 
for ensuring that tests are in fact job-related and non- 
discriminatory. f Given this history, it does not seem 
unreasonable to demand that if examinations are shown to have 
a substantial discriminatory impact on ethnic minority groups, 
some determination be made as to whether they can be justified 
as anything other than arbitrary selection devices. This seems 
particularly appropriate in light of (1) the importance of 
employment to the individual; (2) the fact that the State's 
essential interest is in obtaining qualified employees, a goal

43/
See, e.g., the Guidelines on Employee Selection Procedures 

issued by the United States Equal Employment Opportunity Com­
mission, the agency charged with enforcement of Title VII 
of the 1964 Civil Rights Act, 35 Fed. Reg. 12333 (Aug. 1,
1970) [hereinafter cited as the E.E.O.C. Guidelines). Identi­
cal Guidelines have been adopted by the Office of Federal 
Contract Compliance, charged with enforcement of Executive 
Order 11246 against discrimination by government contractors,
35 Fed. Reg. 19307 (Oct. 2, 1971), as well as by state anti- 
discrimination agencies (see, e. q. , Pennsylvania's Guidelines 
at 1 Pa. Bull. 2005, Oct. 16, 1971, CCH E.P. Guide *5194; 
California Fair Employment Practices in CCH E.P. Guide 
f20, 861; Colorado Civil Rights Commission Policy Statement 
on the Use of Psychological Tests in CCH E. P. Guide f21,060.

43



furthered by a demand for job-related tests; and (3) this Nation's

traditional insistence that employment selection and promotion
44/

decisions be made on the basis of merit.

(2) The Court Properly Determined That Examinations Which Have 
Such a Discriminatory Impact Violate the Equal Protection 
Clause if They Cannot Be Shown to Be Job-Related.

The Examiners claim that the court erroneously ruled that

once their examinations had been shown to have a substantial

discriminatory impact on blacks and Puerto Ricans the burden

shifted to defendants to show a "compelling state interest" in

maintaining such a system, arguing that the court should instead

have applied a "rational relationship" test (Brief pp. 36-41).

44/cndeed, even if defendants' examinations had not discriminated 
substantially against blacks and Puerto Ricans, the court's find­
ing that they could not be shown to be job-related established a 
violation of Due Process and Equal Protection. Schware v. Board 
of Bar Examiners, 353 U.S. 232 (1957); Chaney v. State Bar of 
California, 386 F . 2d 962 (9th Cir. 1967) (dictum) . Thus in Armstead, 
supra, the court specifically held that defendants' uses of the 
GRE violated the Due Process Clause because they constituted 
"arbitrary and unreasonable qualifications for employment" wholly 
"apart from [their] discriminatory aspects" (325 F. Supp. at 570). 
Similarly in Baker, supra, the court concluded that "apart from 
its discriminatory aspects," use of the N.T.E. without a demon­
strated correlation between test scores and job performance violated 
the Due Process Clause. 329 F. Supp. at 722.

Defendants' examination system was also shown to be in viola­
tion of state law, wholly apart from any findings of discriminatory 
impact. The State Constitution provides that civil service appoint­
ments and promotions "be made according to merit and fitness. ..."
(N. Y. State Const. Art. V §6), and N. Y. State Educ. Law §2569(1) 
(1967) provides that the Board of Examiners "shall periodically 
review the validity and reliability of examinations as well as

44



However, while plaintiffs maintain that the compelling state 

interest test is appropriate, this court need not reach that 

issue. The court below rested its decision on findings that the 

Examiners could demonstrate no relationship between their tests 

and the purpose for which they were used, and that indeed the 

Examiners had failed to create job-related examirn tions. These 

findings satisfy the rational relationship test. Thus the court 

found that the Examiners had failed to make out a claim for con­

tent validity, and that available evidence (including the pilot 

predictive validity study) indicated that their examinations were 

not related to job performance. The court therefore did not have 

to reach the issue as to whether the Examiners had the burden of 

providing predictive validity data demonstrating a correlation

between test scores and job performance. Such data would obviously
45/

have been required under the compelling state interest test. 

Application of that test would also have required a showing that 

there was no alternative examination system which would satisfy

44/ (Cont'd)

examination procedures. ..." Since the court found defendants' 
examinations were not job-related and had never been validated, 
they were in clear violation of the State law and Constitution.
These State law claims were presented below, based on the court's 
pendent jurisdiction, but were not decided (App. 140a, 151-53a, 182a).

45/ See, e.g., Arrington, Baker, Robinson v. Lorillard. United 
States v - Jacksonville Terminal Co., all discussed infra pp. 49-51.

45



the state's interest in selecting the best-qualified applicants

without disqualifying disproportionate numbers of blacks and Puerto
46/

Ricans. Again the court below did not find it necessary to

reach this issue. Thus, even if the rational relationship test

urged by the Examiners is applied, the decision below must be
£2/upheld.

In any event, the compelling state interest test is clearly 

appropriate in the circumstances of this case. It is established 

law that wherever different racial groups are differently treated,
48

the burden of justification is placed on the responsible officials,

46/ Under the compelling necessity test a classification must be 
found necessary to accommodate the legislative purpose. See, e.g., 
Kramer v. Union Free School District. 395 U.S. 621, 663 (1969).
An absence of alternative devices to achieve the same end must be 
found. See McLaughlin v. Florida, 379 U.S. 184 (1964). For employ­
ment discrimination cases applying this test see, e.g., Pickens. 
United States v. Bethlehem Steel Co., Robinson v. Lorillard. all 
discussed infra pp. 49 - 5 1 .

4.7 / Compare Western Addition v. Alioto. supra, n. 31 p. 36 , where
the court found unintentionally discriminatory tests placed the 
burden on the employer of "showing a reasonably necessary connec­
tion between the qualities tested in the ... examination and the 
actual requirements of the job to be performed" (330 F. Supp. 539-40); 
Qastro v. Beecher, 4 CCH E.P.D. 17569 at p. 5282, supra n. 31 p. 36 
where the court struck down police examinations on the grounds that 
they had not been prepared by persons with special training in test 
design, did not appear to be content valid, and had not been 
empirically validated.

48_/ This principle was first established in the jury discrimination 
cases. See, e^. , Norris v, Alabama. 294 U.S. 587 (1935), and cases 
cited supra n. 38 . In these cases courts have placed on the
government the burden of justifying significant under-representation

46



and is satisfied only by a showing of a compelling necessity.

Such a test is particularly appropriate where, as here, fundamental
50/

interests of the minority group are at issue.

The Examiners claim that the compelling state interest test 

is appropriate only where there is an explicit statutory racial 

classification, but it is clear that it has not been so limited.

49/

48/ (Cont'd)

of minority groups on juries and have refused to make assumptions 
that blacks are less likely to meet the qualifications (which 
typically require, for example, that the juror be honest and 
intelligent, of good character, literate, and never convicted of 
an offense of moral turpitude). See, e .g., Patton v. Mississippi, 
332 U.S. 463, 468 (1947); Brown v. Allen. 344 U.S. 443, 471 (1953); 
Jones v. Georgia, 389 U.S. 24 (1967).

Similarly, in a testing case, courts cannot assume that blacks 
do less well because they are less qualified but must insist that 
the employer explain and justify the differential passing rates of 
different racial groups. Compare the Supreme Court's conclusion 
in Griggs that tests on which blacks did badly were discriminatory 
since "[b]asic intelligence must have the means of articulation to 
manifest itself fairly in a testing process" (401 U.S. at 430).

49/ See, e .g ., McLaughlin v. Florida, 379 U.S. 184, 192 (1964); 
Korematsu v. United States, 323 U.S. 214, 216 (1944); Bolling v . 
Sharpe, 347 U.S. 497, 499 (1954) (all noting that racial classifi­
cations are 'constitutionally suspect' and subject to the 'most 
rigid scrutiny'); Loving v. Virginia, 388 U.S. 1, 9 (1967) ("very 
heavy burden of justification"); Graham v. Richardson, 403 U.S. 365 
(1971); Kennedy Park Homes Ass'n v. City of Lackawana, 436 F.2d 
108 at 114 (2d Cir. 1970).

50/ Compare Harper v. Virginia Board of Elections, 383 U.S. 663 
(1966); Shapiro v. Thompson, 394 U.S. 618 (1969).

47



It has been found applicable wherever a racial classification

results however neutral the statute may be on its face and how-
51/

ever benign the administrators' intent.

And in the employment discrimination area specifically, the 

courts have applied this standard in reviewing employers' use of 

selection devices shown to have a discriminatory impact, without 

regard to the employers' intent. Thus in Carter v. Gallagher. 

suPra n. 3i , the court quoted and applied the "business necessity" 

standard laid down in Griggs (3 CCH E.P.D. *8335 at p. 6993-205),

.51/ In Dandridge v. Williams. 397 U.S. 471, 485, n. 17 (1970), 
the Supreme Court specifically noted that a welfare statute 
"infected with a racially discriminatory purpose or effect" would 
be "inherently suspect." In Goodwin v. Wyman (3-judge court),
330 F . Supp. 1038, 1040 (S.D. N.Y. 1971), the court found that 
under Dandridge, a welfare regulation with a racially discrimi­
natory effect was “inherently suspect" and required a "compelling 
state interest" for justification. In Hawkins v. Town of Shaw. 
supra, n. 30 , the Fifth Circuit held that a "compelling state
interest" test was required where the results of Shaw's administra­
tion of municipal services were discriminatory even though no 
discriminatory intent was shown, 437 F.2d at 1288, 1292-93. Hobson 
v. Hansen, supra n. 30 , held that "thoughtless" discrimina­
tion placed the burden on defendants of producing "convincing 
justification" 269 F. Supp. at 497, 498. See also Jackson v .
Godwin, supra n. 30 , 400 F.2d at 537; Kennedy Park Homes Ass'n v
City of Lackawana. 436 F.2d 108 at 114 (2d''~Cir. 1970) . -----------

The Examiners rely on Johnson v. New York State Education 
Department, 449 F.2d 871 (2d Cir. 1971), but in Johnson, as in 
Dandridge, supra, there was no racial issue. The only classifi­
cation involved was drawn on the basis of the grade level of 
children. While claims of discrimination against the indigent 
were made, the court found that the real issue was not one of 
discrimination but, rather, of whether the Equal Protection Clause 
guaranteed that indigent children must receive free textbooks.

48



and upheld various aspects of the district court's decision based

on the compelling state interest test (striking down, e.g.,

certain arrest record and education eligibility requirements, 3

CCH E.P.D. 18205 at p. 6681). In Arrington, supra, n. 31,

the court held that tests resulting in unintended discrimination

were "constitutionally suspect," requiring for justification a

"demonstrated correlation between scores ... and ability to per-
52/

form" (306 F. Supp. at 1358).

A similar strict test has been applied in Title VII cases.

In Griggs v. Duke Power, supra, the Supreme Court held that:

The touchstone is business necessity. If an 
employment practice which operates to exclude 
Negroes cannot be shown to be related to job 
performance, the practice is prohibited.
(401 U.S. at 431)

Cases since Griggs have made it clear that for an employer to 

establish a claim of "business necessity" he must make a very

/// in Baker, supra n. 31 , the court struck down use of the NTE
because defendants had not satisfied their "very heavy burden of 
justification, which included the burden of showing a "manifest 
relationship" between scores and job performance (329 F. Supp. at 
721). In Armstead, supra n. 31 , the court held that the GRE's
discriminatory impact on teacher applicants placed a "very heavy 
burden of justification" on the employer to show an "overriding 
purpose." And in Pickens v. Okolona Mun. Sep. School Dist.. M.D. 
Miss. E. Div. No. EC6956-K (Aug. 11, 1971), the court held that 
use of the NTE where it had a discriminatory impact placed the 
burden on the employer to show an "absolute, overwhelming neces­
sity* ••• would have to be shown that [it was] the only way
it was possible to obtain competent and worthy teachers." (Slip 
op. pp. 17-18)

49



strong showing both that test scores are correlated with job

performance and that there is no less discriminatory alternative

available. Thus United States v. Jacksonville Terminal Co.. 3

E.P.D. f8324 (5th Cir. 1971), struck down tests on the ground

that the employer's validity study was inadequate, holding that

— demanded "more substantial proof, most often positive

empirical evidence, of the relationship between test scores and
job performance (p. 6993-149). And in United States v .

Bethlehem Steel Corp., 446 F.2d 652 (2d Cir. 1971), this court
defined business necessity as follows:

Necessity connotes an irresistible demand. To be 
preserved, the seniority and transfer system must 
not only directly foster safety and efficiency of 
a plant, but also be essential to those goals.
If the legitimate ends of safety and efficiency 
can be served by a reasonably available alterna­
tive system with less discriminatory effects, then 
the present policies may not be continued.
(446 F.2d at 662)53/

53/ Robinson v. Lonllard C o m .. 444 F.2d 791 (4th Cir. 1971), 
defined Griggs' business necessity test as follows:

... The test is whether there exists an overriding 
legitimate business purpose such that the practice 
is necessary to the safe and efficient operation 
of the business. Thus, the business purpose must 
be sufficiently compelling to override any racial 
impact; the challenged practice must effectively 
carry out the business purpose it is alleged to 
serve; and there must be available no acceptable 
alternative policies or practices which would better 
accomplish the business purpose advanced, or accom­
plish it equally well with a lesser differential 
racial impact.

50



Since the Examiners failed to demonstrate a correlation

between test scores and job performance, and to show that no

job-related system could be developed which would be less dis-
54/

criminatory in impact, their examination system cannot be 

upheld under the compelling state interest test. However, as 

noted supra, the court need not reach this issue since the find­

ings below satisfied the rational relationship test.

53/ (Cont'd)
★ ★ ★

... It should go without saying that a practice is 
hardly "necessary" if an alternative practice better 
effectuates the intended purpose or is equally 
effective but less discriminatory. Thus, with regard 
to testing, the Equal Employment Opportunity Commis­
sion guidelines stipulate that:

Where technically feasible, a test should be 
validated for each minority group with which 
it is used; that is, any differential rejection 
rates that may exist, based on a test, must be 
relevant to performance on the jobs in question.
29 C.F.R. §1607.4(a) (444 F.2d at 798 and n. 7)

Lorillard further notes that:

... although there undoubtedly are significant 
costs involved in validating tests Griggs requires 
that employment tests be abandoned if not specif­
ically validated as job-related.
(444 F .2d at 799 n. 8 )

54/ The E.T.S. proposal, supra p. 20 , provides a strong indica­
tion that an alternative examination system could be developed 
which would be job-related but would not exclude ethnic minorities 
at disproportionate rates (infra, p. 16a).

51



POINT III

THE COURT'S FINDING THAT DEFENDANTS' EXAMINATIONS 
HAD NOT BEEN SHOWN TO BE AND APPARENTLY WERE NOT 
JOB-RELATED WAS CLEARLY SUPPORTED BY THE RECORD

The Examiners' argument that the court erred in its 
findings as to job-relatedness is based on a wholly distorted 
version of the evidence and of the basis for the court's 
findings (Brief pp. 26-36). While we argue below that the 
court's findings were clearly correct, there can be no question 
that they were supported by extensive evidence, briefly sum­
marized at 4-5,8,10-11,supra, and therefore must be upheld as 
not clearly erroneous.

In the proceedings below the Examiners rested their claim 
for job-relatedness primarily on: (1) a number of research
reports and studies which, it alleged, demonstrated the 
validity of their examinations, and provided empirical data 
showing that test scores were related to job performance 
(see p . 7, supra) and (2) on brief affidavits submitted by 
four experts they had consulted, who based their opinions on 
documents submitted to them by the Examiners (see p.7, 
supra). While relying on the research reports, the Examiners 
refused to produce them until ordered to do so by the court 
pursuant to plaintiff's discovery request. It is only now 
that plaintiffs have had a chance to subject the reports to

52



expert analysis, and the court below has rejected them as 
entirely w o r t h l e s s t h a t  the Examiners argue that these 
reports were not central to their claims for validity (Brief 
30-31). Analysis of the Board's research efforts was clearly 
relevant not only to the issues of content and predictive 
validity but also to the credibility of its other claims 
with respect to the efforts it made to establish content 
validity.

Plaintiffs' evidence on the issue of ]ob-relatedness 
included numerous affidavits by experts who had analyzed, 
inter alia, all documents submitted by the Examiners in sup­
port of their claim of job-relatedness, all their alleged 
research reports, and all the documents on which the Examiners' 
experts had based their opinions?-^-/ The court's decision 
was therefore based not simply on its own lay analysis of

15/ The court found that none of the reports and studies 
even related to validity, content or predictive, with one 
exception - a pilot predictive study which demonstrated no 
correlation between test scores and job performance (App. 
215-16a).

See App. 35-39a, 107-20a, 155-64a. What the Board terms 
the court's "superficial critique" (Brief p. 30) of its re­
search reports was based on an extensive analysis of these 
reports by one of plaintiffs' experts. See App. 158-62a,
App. Ex. 15.

53



reports and examinations, but on consideration of expert 
evidence offered by both sides, as well as extensive brief­
ings on the professional testing literature.-^7

It is clear from the evidence below and the testing 
literature that there are two accepted ways of showing job­
relatedness or validity. One is content validation, which 
depends on demonstrating that the test tasks are comparable 
to the tasks of the job being tested for, so that one can 
assume that test performance will be predictive of job per­
formance. The other is predictive validation (sometimes 
called "criterion-related" or "empirical" validation), which 
depends on demonstrating an actual correlation between test 
scores and job performance.

The Examiners now rely solely on a claim of content 
validity (Brief pp. 26-27). But such a claim can be sustained 
only by a showing that adequate procedures have been followed 
in developing job analyses, test specifications, and, finally, 
related test questions.-^7 Here the court found that the

— / See, e.g., Plaintiffs' Memorandum in Reply to Defendants' 
Memorandum Opposing Motion for Preliminary Injunction, Ind.
Doc. No. 51, filed November 6, 1970.
58/

See generally R. THORNDIKE, Personnel Selection &
Measurement Techniques (1949) at 14, 15, 50; CRONBACH, Essentials 
of Psychological Testing (3d Ed. 1970) at 115-50.

54



Examiners had not in fact followed such procedures on the
basis of a record which included evidence demonstrating: (1)
that defendants had never performed any adequate job analysis
for any position,^/ or made an adequate determination of the
criteria being tested for.-^/ t'>\ .y (2) that tests were designed and
administered by an untrained staff, composed of teachers and
supervisors selected on an ad hoc basis, who were subject to
the supervision of a Board which had only one or at times two
persons on its research staff; ^  and (3) that while some
examination questions seemed superficially related to the
jobs at issue, many could in no sense be considered typical
of or related to 3ob t a s k s . ^  Finally, it is most significant

59/ See pp.4,8,10-1], supra.
|0/ The court cited the Strauss affidavit, referred to in the 
Examiners' Brief at pp. 28-29, merely to rebut the Board's 
claim that the meeting he attended served the purpose of 
helping develop test criteria. See App. 214a, 103-05a.
§1/ See pp. 4 8 supra,
62/

See A££. 217-18a, 36-37a, 114a. while the 
court based its conclusions primarily on the inadequacy of the 
Examiners' procedures, it also considered the examinations 
themselves, including the Nov. 1970 Elementary School Principals' 
Examination, which were submitted to it, and subjected to 
expert analysis and extensive briefing.

55



that the Examiners' own experts "did not conduct the kind of
analyses which could support a claim of content validity"

f) 3 /(App. 157a ') and, indeed, did not even claim content validity
(p. 7 , supra).

Moreover, although the court below did not rely on the 
Examiners' conceded inability to provide any predictive vali­
dation data for its examinations, this in itself required a 
finding that the examinations could not be shown to be job- 
related. Expert testimony below, supported by recognized 
authorities in the testing area, indicated that where tests
are used as devices to predict performance, rather than simply

64/to assess proficiency, predictive validation is essential.—

—  As established by the record below and supported by the 
literature, a claim for content validity cannot be supported 
by a mere comparison of an examination with a list of job 
duties (See App. 162-63a, 157-58a, 112-14a). A. ANASTASI, 
Principles of Psychological Testing 100-101, 104 (3d Ed. 1968).

— ^  See p. 8 , supra.
The court noted in its opinion, App. 206a, the published 

views of Dr. Thorndike, one of the Examiners' experts below, 
to the effect that:

"Whenever a test is being tried for selection of 
personnel for some job specialty, it is most desirable 
that it be validated empirically. Experimental evidence 
is called for to show that the test is in fact effective 
in discriminating between those who are and those who 
are not successful in a particular job." R. L. THORNDIKE, 
Personnel Selection 5-6 (1949) (emphasis added).

See also R. L. THORNDIKE and E. HAGEN, Measurement and 
Evaluation in Psychology and Education 616-641 (1969);
O. G. STAHL, Public Personnel Administration 11 (5th ed. 1962)
These standards, accepted in the testing profession, have been 
incorporated by administrative agencies and courts in assessing

56



The need for predictive validation cannot be avoided simply 
by calling a test an achievement or proficiency test rather 
than an aptitude test, as the Examiners suggest. The Board 
uses its tests in an attempt to select the ablest future 
administrators. This is clearly a predictive purpose, and 
it should be supported with predictive validity data.^' 

Content validation is sufficient only where the test 
tasks can truly be said to be comparable to 30b tasks (and 
even then predictive validation is considered preferable).

64 / (Cont'd)
the fairness of test use in employee selection. Thus the EEOC 
Guidelines on Employee Selection Procedures. 35 Fed. R e a  12333 
at § 1607.4(c) (Aug. 1, 1970), supra n. 43, demand that em­ployers have available:

empirical data demonstrating that the test is predictive 
of or significantly correlated with important elements 
of work behavior which comprise or are relevant to the 
job or jobs for which candidates are being evaluated.

As noted supra n. 43, identical standards have been adopted 
by the OFCC and by state anti-discrimination agencies. Cases 
incorporating or approving the EEOC Guidelines include, in the 
public employment area, Carter, Baker, and Armstead, supra n. 31 .
Arrington_v. Massachusetts Bay Transportation Authority,
suPra n.31, adopts a similar standard. The Supreme Court 
approved the guidelines in Griggs, supra, as have other courts 
in Title VII cases, e.g., united States v. Jacksonville Termi- 
nal Co., supra p .50 ; Hicks v. Crown Zellerbach" supra n~. 32.

See A. ANASTASI, Psychological Testing 120-121 (3d ed. 1968)

57



A typical example of reliance on content validity is use of 
a typing test to select typists: since the test task is
essentially the same as the job task, the assumption that it 
is predictive of job performance may be warranted. Defendants' 
tests, by contrast, cannot be considered adequate samples of 
the jobs to be performed. As indicated in the court’s 
opinion, many of the questions are not even superficially 
related to the jobs at issue (App. 217-18a, 232-33a, n. 23).
And even those questions which are superficially related 
cannot be considered equivalent to the tasks the person 
would actually have to perform on the job.

Moreover, the court's finding that the Examiners' only 
attempt at predictive validation "showed that there was little 
or no correlation between success on the tests and job success," 
(App. 216a, see also App. 159-61a), indicates that the Board 
not only failed to show job-relatedness, but that the tests 
were in fact not job related.

58



POINT IV

THE GRANT OF PRELIMINARY RELIEF, BASED 
UPON THE COURT'S FINDINGS WITH REGARD 
TO IRREPARABLE INJURY AND THE LIKELI­
HOOD OF SUCCESS ON THE MERITS, MUST BE 
UPHELD AS A PROPER EXERCISE OF DIS­
CRETION

The trial court's decision to grant preliminary relief 

can be reversed only if it constituted an abuse of discretion. 

And appellate courts should be reluctant to reverse in such 

cases because of the

"flexible interplay between the likelihood of 
irreparable harm to the movant and the court's 
belief that there is a 'reasonable certainty1 
that the movant will succeed on the merits at 
a final hearing. . . . "
Packard Instrument Company v. Ans. Inc., 416 
F. 2d 943, 945 (2d Cir. 1969) 61/

Appellate courts should be particularly reluctant to substi­

tute their judgment for that of the court below where, as 

here, the decision rests essentially on factual findings with 

respect to discriminatory impact and job-relatedness. These 

f indings cannot be disregarded unless clearly erroneous. In

— / United States v. W. T. Grant Co., 345 U.S. 629 (1953); 
Societe Comptoir v. Alexander's f)ept. Stores, 299 F. 2d 33,
36 (2d Cir. 1962); Packard Instrument Company v. Ans. Inc., 416 
F. 2d 943, 945 (2d Cir. 1969).

Thus where, as here, the court finds the balance of hard­
ships tips toward the moving party, he may obtain relief "if 
he has raised questions going to the merits so serious, sub­
stantial and difficult as to make them a fair ground for 
litigation and thus for more deliberate investigation,"
Checker Motors Corp. v. Chrysler Corp., 405 F. 2d 319, 323 
(2d Cir.) cert. denied, 394 U.S. 999 (1969); Pino de 
Laurentiis Cinematografica, S.P.A. v. D-150, 366 F.2d 373,
375 (2d Cir. 1966).

59



the instant case they were fully supported by the record.

The injunctive order issued was hardly "sweeping" and 

"unprecedented"as characterized by the Examiners. It was 

carefully drawn on the basis of a series of proposed orders 

and supporting affidavits, exhibits and memoranda. It was 

clearly reasonable, — / and consistent with orders issued in 
comparable cases.

The court's findings as to irreparable injury and the 

balance of hardships were clearly correct (See App. 223-25a, 

255a). Denial of the injunction would perpetuate racial dis­
crimination and deny plaintiffs and their class any opportunity 

for appointment throughout the undoubtedly lengthy period until 

this case is resolved on the merits; and opportunities for

Its prohibition against use of lists based on examinations 
covered by the survey was not disputed by any party below.
Its prohibition against use of lists not yet promulgated when 
this action was instituted is obviously reasonable in light 
of the court’s refusal to bar further examinations at that 
time. And in prohibiting future examinations the court pro­
vided defendants full opportunity to obtain a modification 
to permit implementation of a new examination system (App. 259a).

Compare, e. q . , the orders issued in Carter v. Gallagher, 
supra, 3 CCH E.P.D. f8335, p. 6993-201, and Hicks v. Crown 
Zellerbach, 319 F. Supp. 314, 321 (E.D. La. 1970), both bar­
ring future examinations until such time as they had been 
validated in accordance with E.E.O.C. Guidelines.

60



appointment will then be limited since positions will have
been filled. In addition, hundreds of minority acting
supervisors, selected by local boards on the basis of their
ability to perform, would lose their positions by operation

71/of law if relief were not granted. By contrast, those
persons who are or would be on the eligibility lists have an 
equal opportunity, under the court's order, to compete for 
positions on an acting basis. Since tenure has been elimi­
nated for supervisors they will lose no tenure rights. And 
if defendants ultimately prevail on the merits these persons 
will have ample opportunity for appointment to all the positions 
filled by acting personnel during the interim.

Finally, the court's finding that no harm would result to

— 7 The fact that tenure for supervisors has been eliminated 
does not mean as a practical matter that such positions will 
be available since traditionally virtually no one has been
discharged even during so-called probationary periods.
71/— * The court found, for example, that there were 131 
minority acting Assistant Principals (App. 223a). The 
Examiners' argument (Brief p. 45) that only a few people would 
be threatened with the loss of their acting positions is ab­
surd since it assumes erroneously that the only position in­
volved in the instant action is the Elementary School Principal 
position. In fact it is clear that every supervisory position 
is covered and that there are hundreds of acting minority super­
visors in the system.

61



the public (App. 225a, 255a) was obviously warranted by its 
conclusion that the Board's examination system was not job- 
related. It was further supported by the facts: (1) that the
Chancellor considered the examination system unworkable; (2) 
that he and the Board of Education, the officials responsible 
for the administration of the City's schools and appointment 
of personnel, have chosen not to actively participate at any 
stage of the proceedings,^ including this appeal; (3) that 
acting appointments have been a tradition in the school system, 
constituting a large percentage of total appointments, and (4) 
that in the past appointing authorities have depended in large 
part on an acting appointment system to obtain the most quali- 
fied personnel. Since no other City School System in the
State has an examination system like New York's (App. 184-85a), 
it is hard to imagine that much harm would result from its tem­
porary suspension, particularly since virtually every group 
which has studied the Board of Examiners has concluded that it
should either be abolished or its procedures radically re- 

74/formed.

72/—  The Board of Education's participation was limited to 
opposing the T.R.O. at relatively early stages in the pro­
ceedings below; it did not oppose entry of the preliminary 
injunction after the court's decision.

— / Indeed the Board of Education had in the past approved 
the practice of making acting appointments as a means of in­
creasing the number of qualified blacks and Puerto Ricans in 
the supervisory ranks. Exhibit 2 to plaintiffs' motion, Ind. 
Doc. No. 2. And acting appointments pending development of a 
new examination were approved in C.S.A. v. Board of Education. 
23 N.Y. 2d 458, 29^ NY Supp. 2d 547, 245 n.2 2d (1969). That 
case also held acting principals qualified on the grounds 
they had satisfied state certification requirements.
74/ See authorities cited n. 20, supra.

62



Moreover, developments since the court's decision 
demonstrate that the decision will not only result in no 
harm to the school system but, indeed, may result in the 
first significant reform in examination procedures in the 
Board's history. in the meantime a workable interim selection 
procedure has been devised, which guarantees that acting 
supervisors will possess adequate education and experience 
qualifications, and will be selected on the basis of merit 
in accordance with regularized procedures.

CONCLUSION

the reasons stated above, the district court's 
decision granting preliminary relief should be affirmed.

Respectfully submitted,

JACK GREENBERG
JONATHAN SHAPIRO
ELIZABETH B. DUBOIS 

10 Columbus Circle 
New York, New York 10019 
Phone (212) 586-8397

GEORGE COOPER
435 West 116th Street 
New York, New York 10027

MICHAEL 0. FINKELSTEIN 
26 Broadway 
New York, New York

Attorneys for Plaintiffs-Appellees

63



APPENDIX A

Ka y e .Sc h o l e r . Fierman Hays S, Ha n d l e r
•42 5  P A R K  A V E N U E  N E W  Y O R K  N Y 1 0 0 2 2  

i2\2' P l a z a  9 8 4 0 0

HAROLD l r i fRMAN 
MM 1 ON HANDLER 
FRLOfW CK R LI/iNCSTON 
Stani t y D wAxur nG 
MILTON KJNEN 
NATHANIEL H JACKSON 
R'CHARD C ‘ l FhC h 
JOSEPH C* CONNOLLY
Sidney j su b e r m a n
FREDERICK GLIDE WG 
JAY O. KRAMER 
STANLEY P . ROBINSON 
S'DNEY A DIAMOND 
FRED A H-i ' A n

NORMAN S'NHMCh
DONLN G i t  "*
J O S H U A  F  G* ; ■ ' H E R

P f  TER M c s -- P- N
^ A A S l V . i  N t W ' . - A N  PETER v.
S L ' M -  .? ■ ' -  . A • E N 
3A u , j  o PLAN 
DA'. O' GOLL0F «?L M t‘ . 1 Ml
B E "  P A M  A • - M

■■ ■ r g r j r  ■*! i-fr~i~ y~ 1' r n r r r r ^
SAUl l c  •* N 
MM TON R A t ST I 
B W NIM K l N 
Wit l 1AM J ISAAC TON 
SHCt DON O' M N' IS 
FRED N FISHMAN 
FREDtRiC.K M DiM.LEN 

-  GERALD FI l l E R
ARNOLD i . Gin-DDERG

Lfl'IN A : S< MAN 
SIDNt ' *■ At L ' t .
S'AN  £ I R( . EN S 
ROBER’ M SC mAn .’ i r  
JAY G STRJM

November 9, 1971-
J A COB SCH O L E R  
JAMES S. HAYS

COUNSEL

EUROPEAN OFFICE 
4 4  C H A M P S  El > SEES 
PARIS V»l .FRANCE 

TEL i t5 88'0

c a b l e  a d p w e s s c s  
KAYEMACLEFV n e w  YORK  

KAYEMAC lER PARiS

y Cl I * NUMBERS 
N E W  YORK 234060 

PARIS 6598i

Elizabeth B. DuBcis, Esq.
10 Columbus Circle, Suite 2030 
New York, N.Y. 10019

Dear Miss DuBois:
As I told you i n our telephone conversation of 

yesterday, the Board of Examiners has been diligently at 
work on a proposal for modifying its supervisory examination 
procedures. 
findings and 
waives none < 
believes tha' 
fulfill its 
it. would be 
visory posi 
pending t:o!

at it has a n
+- t ■ i t  o r* r*

f q i t r H . . oa a  -l  i L/ O' . * —
tions to U 0  J.

. *_ ... >Y , r- f

ccordingi y ,
t prejudi C c  ,

î3.rd sl r » Q  p giy d
of T  i l  , T O LA a Y  Po Mans •rie
S by LS Lx rsu ing th i
oi 1 iga1 * c r. as a P
net:on. ■: e E;Car ' YD

j the ent, 
ur appeal

A. d .  J- I J -L -L

”* 0 0 x am _l i j

not feel 
for super- 
s y s t em

is to forward to 
il for modifica-

.y, the Board has asked
+- o  t  t  c  c  r  o  ^  >•* v » T  r*r -o  <Ty : - D" di- l U L  - L  d  p  i p  v-y .

tion of supervisory examination procedures. We hope you will 
agree that tne proposal. reflects a orairiauic change in the 
entire examination approach.

It is cur further hope that this proposal can be­
come the basis for a joint application to Judge Mansfield 
for a modification of the preliminary injunction.

Very truly yours,

Z. Cohen
9 r7 ( '  • q  \

e r. c .

BY HA:

la



BOARD OF EXAMINERS

Proposal for Modification of 
Supervisory Examination Procedures

Pursuant to the final paragraph of the preliminary

injunction issued on September 20, 1971 by the Honorable Walter 

R. Mansfield of the United Stages District Court for the South­

ern District of New York, in Chance, Mercado, et al . v. The Board 

of Examiners and the Board of Education of the City of New York, 

et a 1., the Board of Examiners is applying for modification of 

the preliminary injunction to permit further examinations for 

supervisory positions and the promulgation of lists, issuance 

of licenses, and making of appointments based thereon. The 

application is made without prejudice to all rights on appeal.

It is made on the basis of a proposal for new examination pro­

cedures that have been developed during intensive meetings involv­

ing the following experts:
Dr. Roscoe C. Brown, Jr., Professor of Education and Director 
of the Institute of Afro-American Affairs, New York University

Dr. Harry Gilbert, Professor, School of Education, and 
Coordinator of Urban School Psychology Programs, Fordham 
University

Dr. Nathan Jaspen, Chairman, Department of Educational 
. Statistics, New York University

Dr. Gerhard Lang, Chairman, Department of Educational Research 
and Professor of Psychology and Education, Montclair State 
College, Montclair, New Jersey

Dr. Robert Linn, Head of the Department of Research and 
Development, Educational Testing Service, Princeton, New 
J ersey



3a

Dr. Frederick J. McDonald, Chairman, Teacher Behavior Group, 
Educational Testing Service, Princeton, New Jersey

Dr. Joseph L. Moses, Psychologist, American Telephone and 
Telegraph Company

Dr. Robert D. North, Associate Director, Professional Exam­
inations Division, Psychological Corporation, New York,
New York

Dr. Benjamin Rosner, University Dean, Teacher Education,
City University of New York

Dr. Marvin Sontag, Associate Professor of Psychology and 
Education, Teachers College, Columbia University

Dr. Robert L. Thorndike, Richard March Hoe Professor of 
Psychology and Education, Teachers College, Columbia University

Dr. E. Belvin Williams, Associate Dean for Administration, 
Teachers College, Columbia University, (formerly Northeast 
Regional Chairman of Association of Black Psychologists)

The proposal concerning new examination procedures 

represents the considered professional judgment of the experts 

who participated and is intended to gain the widest possible 

support for the Board* s procedures and improve their compatibility 

with the decentralised school system administration. These experts 

not only were given an opportunity to review past examinations 

(written tests, oral tests, performance tests, and conference tests), 

but they also brought to the deliberations expertise in a variety 

of areas: psychometrics in education and in industry, development

and-administration of teacher and supervisory educational prepara­

tion programs, analysis of teacher and supervisor behavior in the 

school setting, and knowledge of the current status and thinking 

of minority groups witti respect to testing in general and testing 

for teaching and educational supervisory licenses in particular.



4a

The proposal deals with new procedures in a compre­

hensive supervisory license assessment program in four areas: 

job analysis; job-related examinations; standards and reporting 

of results; validation of examinations.

1. Job Analysis

Proposed Procedure' " " —— y,

The Board of Examiners will request from the Chancellor, in 

addition to statements of duties of the position, appropriate 

skills, qualities and behaviors, in other words, specific job 

analyses for each position for which it is required to conduct 

|examinations. Following the consensus of the experts consulted, 

the Board of Examiners will develop specific criterion measures 

by devising appropriate means of identifying essential job 

behaviors and by consulting actual logs of incumbent supervisors 

7 of critical incidents on the job as defined by John C. Flanagan.

These will reveal what the person must do on the job, what per­

formances are of critical importance and what actions differ- 

'entiate between the well-perceived and the inadequate individual 

in this job setting. In addition, specific perceptions of the 

job will be obtained from Community School Boards, parents and 

community superintendents and these will be incorporated into 

the criteria for selection of the supervisors for whom the tests 

will be given. In addition, research literature which supports 

the relationship between specific knowledges and skills and 

specific criteria of effective supervisory or administrative



\>a

performance will be consulted to identify further the qualities 

to be assessed. This procedure will provide a broader empirical 

basis for the job-related examinations to be developed.

2. Job Related Examinations

Proposed Procedure

Job relationship of tests will be established by closely 

matching tested knowledge and skills to the essential critical 

behaviors required on the job as identified. The language 

mastery element in written tests will be deemphasized to take its 

place alongside such data as are obtained through other tests 

which reveal essential elements of administrative ability, super­

visory ability, attitude toward tne learning process for the vary­

ing student populations of the city, human relations skills and 

attitudes, and such essential additional skills as the candidate 

may offer, e.g., knowledge of spoken Spanish, knowledge of Afro- 

American and Hispanic culture, etc. The job history and experience 

of applicants will be reviewed for relevance to the particular 

'license being sought. To illustrate, significant elements of 

administrative ability can be tested without recourse to a written 

examination by having a candidate defend orally the actions he 

would take on a number of realistically simulated documents he 

would be faced with in his "in-basket" on the job. Similarly, 

important aspects of supervisory ability, that is, the ability 

to observe a lesson and to help the teacher improve it, could be 

tested by having the applicant observe an actual lesson or a



6a

video taped one, and then evaluate it orally before a panel.

Group interviews may be used to assess an individual's performance 

in relation to other members of the group. Video-taped simula­

tions of critical incidents involving significant human relations 

skills may be used to appraise an applicant's ability to handle 

a critical situation. In every instance the test task would be 

carefully related to the job behaviors and skills which make up 

the criterion derived from job analysis and study of available 
research.

3. Standards and Reporting of Results

In the past, specific weights were attached to each one of 

the battery of tests with a few exceptions (written English, 

physical and medical, and record were rated either satisfactory 

or unsatisfactory). A candidate had to achieve a weighted 

average of 60 per cent on the battery of tests to pass. If a 

candidate secured less than 50 per cent on any part of the exam­

ination, he was eliminated. If he obtained 50 per cent or more, 

'he was continued in the examination but he had to compensate for 

marits lower than 60 per cent because he had to achieve a weighted 

average of 60 to pass. In many instances, the written test was 

weighted at 40 to 60 per cent of the total grade.

Proposed Procedure

The proposed procedure is made possible by a provision of the 

1969 Decentralization law which requires the preparation by the



, a

Board of Examiners of lists of eligible supervisors through "open 

qualifying" examinations.

In this procedure, a series of job-related tests will be 

retained but the results will be reported in a completely differ­

ent manner. A profile of performance will be available to the 

Community School Boards for all qualified candidates. This profile 

will present an assessment of 'she performance of all candidates 

in all of the tests without specific weight and without exact 

numerical grades. A sample profile follows on the next page.



STEP
ONE

STEP
TWO

8a

Sample Assessment Profile

(The ter.to indicated are merely 'examp]or. of those to be 
used, but total ace or, ament is not restricted to these.)

a;5 C,o Pc Mp H*1—*H* H*0)H* CifD
ft 1Understanding of Essential Principles

of Administrative Behavior
(as measured by "in-basket" situations)
Ability to Evaluate
Observed Classroom Teaching and to
Effect Improvement
(as measured by response to actualteaching)
Understanding of Human Relations Prin­
ciples Involved in School Management 
(as revealed by reaction to taped 
situations)
Understanding of the Educational Pro­
cess in the Urban Setting 
(as revealed in a written test on edu­
cational literature - for entry level 
as chairman or assistant principal)
Communications Skills: Ability to 
Communicate to Staff, Students, Community
(as revealed in simulated tasks)
Skill in English Communication 
(as revealed in a writing sample - for 
entry_ level as chairman or assistant principal)
Physical and Medical Test

e>
Record, Including character and evalu­
ation of past experience 
(including fingerprint check required 
by Board of Education)
Special Skills Tested at Applicant's 
Request (e.g. knowledge of Spanish) <>

>
Special Experience 
(identified from vita supplied bv 
applicant and verified by the Board 
of Examiners)
Performance on the job 
(by team including professionals and 
representative of Community School Doarc 
at request of Community School Board foi 
principals' licenses)

c
DM 1 
O 0 1 p
2.1!" :.i •:

v •c» ... l.
O'! ' ; 
( :

Qualified



9a
<

It can be secnlrom this sample profile that, though 

the candidate is qualified overall, he is unqualified in one of 

the areas tested. In the event that the total performance of 

the applicant results in a rating of "unqualified" in two or more 

areas, he would be rated unqualified and not recommended for 

licensing and a place on the qualifying list. Qualifying scores 

would be set at levels that would insure minimum competence 

necessary to meet the performance criteria on the job. These 

would be arrived at after thorough professional review of exist­

ing research concerning the skills tested and professional judg­

ment as to minimum levels of acceptable performance. Upon receipt 

of the list of qualified eligibles, together with their assessment 

profiles, each Community School Board could evaluate the indicated 

achievement levels of each eligible in the light of its own needs 

attaching more importance to some aspects of the total assessment 

and less importance to others. Thus, a candidate who did poorly 

in English communication but well ir. administration and super­

vision might be more acceptable to one Community School Board than 

to another. Another aspect of the profile would be the evaluation 

of additional skills claimed by the applicant; thus, an applicant 

who claimed fluency in Spanish would be tested for that skill and 

this additional qualification will be reported together with the

other aspects of his assessment.
The Board of Examiners would undertake to provide a

manual to help Community School Boards interpret the results



10a

reported and would consult upon request with any Community School 

Board in matters of personnel selection. Each Community School 

Board would then submit to the Board of Examiners a list of quali­

fied persons in whom they were interested for principalships for 

an assessment of their performance in the job they now held. 

(Practical considerations of available experts and cost factors
V

preclude performance evaluation of all eligibles in some exam­

inations where there are large numbers of applicants.) The Board 

of Examiners, after making such assessment, would evaluate each 

eligible as to current performance in the light of criteria 

cooperatively arrived at by the Board of Examiners and the 

Community School Board involved. Accompanying the profile would 

be descriptions of the performance of the qualified applicants 

to enable the Community School Boards to have as full a knowledge 

as possible of the qualifications of the applicants in question. 

Implicit in the assessment program procedures are the same safe­

guards that are currently in effect: professional selection and 

training of the examination assistants who construct and administer 

the individual tests and the right of the applicants to full dis­

closure of the guidelines used in the assessment process and the 

right to appeal from any or all of the findings.

ty. Validation

Proposed Procedure

The matter of validation is an extremely complex one. This 

was recognized by the Court in the extensive discussion of the



11a

problem in its Opinion (pp. 27-35), and in a memorandum 

accompanying the preliminary injunction in which the Court urged 

that the parties show "a willingness to yield to the public

interest at stake rather than to insist on soecific criteria

that may be impractical or unrealist ic wnetTier~in the field of 

predictive validity or otherwise." The difficulties of valida­

tion are further illuminated by the decision in V .0. v. Georgia 

Power Co., 3 FEP 767 (U.S.D.C.-N.D. Ga. 1971) wherein the Court 

observed: "According to the government's experts, it would re­

quire at least two years to develop a testing program to meet all 

the criteria for validity." 3 PEP at 780. The Court there went 

on to note (3 PEP at 7°7 fn.3):

"However, the rather startling evidence offered 
by the government was to the effect that there 
was no test known to exist or yet devised which 
could meet such standards (of the Equal Employ­
ment Opportunity Commission). The Court con­
cludes that Section l was net intended by
the Congress nor Inter;reted by the Supreme Court 
in such meaningless- fashion. • Psychological test­
ing is itself a now inexact science. Reputable 
able people (her-.-,- two former partners) readily 
disagree over the ultimate validity of almost any 
test when related to a particular job or employer. 
In Grir-ns, validity is a reasonable, not absolute, 
requirement. It is inescapable that the Equal 
Employment Opportunity Commission regulations, 
while on the whole helpful, are not binding in 
this respect. See the common sense approach taken 
in United States v. H.K. Porter Company, 296 F. 
Supp. lid at 7^, 79, 76, 1 PEP Cases 615, 70 LRRM 
2131 (N.D. Ala. 1*68)."

With these comments in mind, it is proposed that the 

new procedures supplement prior efforts in establishing content 

validity with an additional test of validation: an on-the-job



performance evaluation by the Board of Examiners of supervisors

two years after1 they have been in their new job. This performance- 

based evaluation would take into consideration (a) the criteria as 

to job behaviors previously obtained from supervisors and admin­

istrators on the job and (b) feed-back from the Community School 

Boards and top-level supervisors as to evaluation of the per-
V.

formance of appointed eligibles. Roth sets of data would then 

be compared with the original assessment to result in a valida­

tion of these original tests. Such evaluation, would serve two 

purposes: to enable validation of the original assessment and

to provide an impartial evaluation of the incumbent’s performance. 

This on-the-job performance evaluation would enable follow-up of 

significant numbers of licensed supervisors. Evidence obtained 

from these performance-based evaluations would make available data 

enabling improvement of the assessment program by retaining those 

parts which correlated best with performance and eliminating those 

which correlated worst.

'Conelusion

The Board of Examiners is presenting this proposal as 

the consensus of the experts consulted who evinced a remarkable 

degree of agreement. It represents a conscientious effort to 

take yet another hard look at its procedures and to make them 

more flexible and adaptable to the Community School Board type 

of administration. It does so without abandoning the essential 

elements of a merit system which protects the rights of all



13a

applicants. 

the ultimate 

varied comma

Most important it takes cognizance or the needs 

beneficiaries of this action, the children in tho 
ities of New York City.

V



APPENDIX B

Memorandum for

Subject: Proposal to N.Y.C.

MR. BARRETT 
MU. COHi.N
MR. COOPER
MS. DUBOIS 
MR. GREENE 
MR. SCH/.PIRO 
MR. ROCKOVITZ

From: Frr.derick J. McDonald

Date: December 21, 1971

Attached is a copy of the draft of our proposal. After I have 
discussed this draft on December 27th, I will prepare a final copy.

I have omitted the budget, but will bring a preliminary draft 
to the meeting on the 2 7m. The final draft will include Appendices and 
vitae of the personnel who will work on this project.

This draft copy omits points that I think should he discussed among 
the parties involved before and if they are to be included in the proposal. 
Please keep this in mind as you read the proposal.

14a



15a

PRi: I'ACl.

This Jov unu-nt describes t !.e general methods and worx plan to he 

used by ETS to develop a prototype assessment system for the selection of 

candidates for an administrative position in the New York City Schools.

It was thought to bo unnecessary to describe these procedures in detail 

since they are well known.

It is impossible to predict in this proposal the exact character of 

the assessment system to be developed. Rather, the proposal describes 

the empirical methods that will he used and out of which an assessment 

system will evolve.

The proposal commits the developers, ETS, only to create a system 

that is non-discriminatorv and job relevant. There is reason to be 

optimistic that the plan described here is likely to produce such a system.



16a

INTRODUCTION
This proposal describes a plan for developing a prototype assessment 

system for selected administrative or supervisory examinations to be used 

by the Board of Examiners of New York City. The purpose of developing this 

prototype is to create a model of an assessment system that is job relevant 

and non-discriminatory where job relevance is the primary criterion but 

where its application does not systematically or inadvertantly exclude 

individuals on the basis of race, sex, religion, and other non-merit factors. 

"Job-relevant" means that the assessment procedures will identify those 

individuals who have the knowledge, skills, and personal characteristics 

which can be shown to be directly related to effectiveness in the position 

for which a selection is being made. "Non-discriminatorv" in this instance 

means that the assessment system will select or reject members of ethnic 

groups in proportion to their numbers in the applicant group.

"Prototype" means an assessment system which may be used as a model 

for developing evaluation procedures for selecting individuals for other 

positions, and which when approved may be used in the examination system 

of the Board of Examiners.

The product of the plan proposed here will be: 1) a description of

the knowledge, skills, and personal characteristics required for the position 

for which the assessment system is being developed; 2) a description of 

the criteria local boards will use in choosing among candidates for the 

position; 3) a set of assessment procedures or techniques; 4) alternative 

plans for using these procedures so that they will provide the information 

the local boards need to make selections; 5) data on the non-discriminatory, 

job-relevance, and other psychometric characteristics of the alternative 

procedures; 6 ) cost estimates for the use of the procedures; 7) a set of



17a

recommendations on to!low-up methods to be used to gather data on the 

effectiveness of the procedures in selecting successful administrators 

or supervisors.

Tasks and Methods

In this section we describe the tasks to be accomplished and the 

methods to be used in accomplishing them. hater sections describe the time 

schedule of the plan and the personnel who will work on this project.

Job Analyses: Extensive job analyses will be made of the position in

selected school districts. The following methods will be used:

1) ETS staff will follow the incumbents of the position through their 

daily routine to determine the nature of their work. A sample of days over 

a two month period will he drawn to select the days on which ETS personnel 

will be present. ETS personnel will "follow" the incumbent of the position 

through his or her daily schedule (with minimum interference with the 

incumbent's duties), being present in as many situations as is possible 

without interfering with confidentiality in interpersonal relations or 

accomplishment of the task at hand.

The person being observed will be fully informed of the nature and 

purpose of this activity and will be encouraged to be co-worker of the 

observers. They will be asked about their choice of methods to cope with 

the problems that arise and the bases of their decisions.

2) An analysis will be made of all the correspondence, telephone calls, 

and face-to-face requests made of the incumbent. When these materials and 

conversations are confidential, the incumbent will be asked to summarize

thei r content and his de • i sLons aid act ions.

3) The incumbents wi11 he .skec to keep a Ion of t lie i r activities

duri ng the periods when ETS staf:: are not present.



18a

and

4) An analysis will bo made bv KTS personnel of the legal, policy, 

procedural statements governing the position beins studied.

5) ETS personnel will make an analysis of documents describing what

the position "ought" to be.

6 ) ETS personnel will interview teachers, community superintendents, 

community board members, and representatives of major educational organizations 

such as organizations of school administrators and parents' groups about 

theii perceptions of what the position requires and the characteristics 

the incumbent ought to have.

A description of the position will be drawn from these observations 

and analyses. This description will include a list of the tasks the incum­

bent must successfully accomplish, the decisions he must make, the characteristic 

actions or methods he must use, the knowledge, skills, and personal traits

presumed to be required to accomplish the tasks, make the required decisions, 

and successfully execute the actions needed to solve the problems the 

incumbent will face.

This description will be used to specify the characteristics of the 

assessment system to be developed.

— -h — Develoj^ment^: It is impossible to specify in this proposal the exact

nature of the testing techniques that will be used and the assessment system 

that will be proposed. It is expected that the techniques will include 

a wide range of procedures such as paper and pencil tests and performance 

tests. It is to be understood^ however, that greater effort will be given 

to the development of techniques that measure, ability to perform effectively 

i_rwlyc_ • it should also be understood that the system developed

will use a variety of procedures w h i c h  will he included only if they can 

be shown to be correlated with ef-'cct ivt i,ss in the position.



19a

The assessment system will nr u, ,ci,. k
11 ,robably be a set o i successive steps

through which the candidate willwiu pass. His performance on each phase

will be appropriately weighted; and his weighted scores will kscores will be presented
in a profile.

l e ^ L j r ^ O u t - .  Each of the procedures developed will be administered 

to a sample of candidates. The samples will be determined in the course

tHlS Pr°ieCt‘ Ir may be assumed that the size of these 

samples will vary; for example, if a paper and pencil test is developed 

first phase, a large number of candidates will take it.

“~ ~ : lhe Pr°Cedures developed will be validated by testing
a sample of current incumbents of the position who will be rated on

I SUCCeSSfUl j°b perfo™anee. Performance on each procedure will be 

correlated with measures of on-the-job performance.

Advisory Beard

ETS wi n  request individuals knowledgeable about the position and

Che Pr"ble“  °£ Vork «flr schools to serve in an advisory eapaoin

P icet stafi Thrs Advisory Board will recommend problems to be 

anticipated and procedures to be followed in the development of the 

assessment system. The Board will review progress and final reports.

The advisory Board win meet in the first month of the project. At 

this time the Board will be presented the sampling plan for selecting 

districts and schools within which the job analyses w i n  he made, a des­

cription of the work plan for condoning the Job anaiyses, and a description

of the staff and an opportunity to , horn. The Board w i n  ho asked to discuss
the problems to be . lf A ...

L ....."citing the job ana lyses and interpreting
the information Lh.it w i l , be gathered.



2 0a

At this i.ine the 5 lan and stafi for interviewing the Community 

Superintendents and Boards will also be presented to the Advisory Board 

for their review and advice.

The next meeting of the Advisory Board will be held after the job 

analyses and interviews with Community Superintendents and Boards have 

been held. LI'S personnel will present summaries in the form of job des­

criptions which state the performances required of th. incumbents of the 

position for which the assessment system is being prepared. The function 

of the Board will he to make a critical analysis of th.is description. At 

this time EiS will also present the specifications for the assessment 

system which will be reviewed by the Board.

« The Board will meet for the third time after pilot forms of the

components of the assessment system have been developed. The Board will 

review these components with specific attention to their relation to the 

job descriptions developed in the first phase of this project. The Board 

will be used as a panel of experts to judge the content validity of the 

components 01 the assessment system.

The last meeting ol the Board will take place when a preliminary draft 

of the final report is available. The Board will be asked to react to 

this report by criticizing the analysis and interpretation of the data 

and the recommendations which will have been made.

The above list of tasks to be undertaken by the Advisory Board suggest 

the criteria for the selection of its members. The Board must have technical 

expertise in assessment and evaluation, familiarity with the problems of 

administering New York City Schools, sensitivity to the social, political, 

am! r’jlt’ir',i ! ' r f * ‘a1 • •mmtnlt i«..s and >• •..•>'Is . • New York City, and

commitment to helping to produce an assessment system which is non-discriminatory

and job relevant.



21a

The Role of the Community Boards

We have provided for extensive consultation with the Community 

Boards and their Superintendents. This consultation will serve three 

purposes.

First, the Boards and Superintendents will describe the performances 

and characteristics required for the position being studied as they perceive 

them. The members of the Boards will be asked what the incumbent should 

be able to do, and what they see are desirable characteristics for the 

incumbent to have.

Second, when the desired performances have been described by the 

processes described above, the Boards and Superintendents will be asked to 

react to these descriptions and to weigh their relative importance. A 

description ol the criteria and their relative and comparative importance 

will be prepared from this analysis.

Third, the Boar.Is and Superintendents will be interviewed to determine 

what information they need to assess the potential of a candidate for a 

position in their description. They will be asked to review the instruments 

developed and their scoring procedures to determine if they are likely to 

provide the information required. From this analysis a description of the 

information system to be used with the assessment system will be developed.

Construction of Test Procedures

ETS personnel will construct the specific procedures to be used. F.TS 

has developed a written examination for Chicago School District which meets 

the requirement of being »on-discriminatorv. A copy of a report on this 

examination is appended to this proposal fsee Appendix B). Materials similar 

to this tyamin.it . • n will be do.eloped, it is understood t nut ETS will not use 

tests developed t i other c t ten ts vitnout revising them to make them relevant

to positions in New York City.



2 2 a

E1S has also cIcvl loped many Ji* t«. rent forms o 1 a performance measure 

called an in -basket" tost, from trie tray on the desk of an administrator 

in which is deposited the incoming mail. This test is a simulation procedure 

in which the candidate assumes the role of an administrator in a prescribed 

setting and the candidate reacts to "real" problems. His decisions and the 

way he carries them out, as if he is actually on the iob, provide a sample 

of his administrative behavior. This technique has been used widely in business 

and government and performance has been shown to be related to administrative 

effectiveness.

The Teacher Behavior Research Group of F.TS has developed simulation tests 

of supervisory skills. In these tests supervisory personnel watch videotapes 

of classroom teaching on which thev comment. They are scored on what they perceive 

as deficient or commendatory, and on the recommendations they would make.

These are some of the kinds of procedures likely to appear in the proto­

type of the assessment system to be developed. Technical analysis will determine 

what will be included in the prototype to be proposed.

Alternative Systems and Their Costs

It is likely that more than one system can be developed. These alternatives 

will differ in the components included, their relative weights, their predicative 

validity, their relative utility lor different purposes, the order in which the 

components are organized and their comparative costs. We will propose several 

such alternatives, relevant cost informal; ion, ana their comparative utilities.

We regard the choice of a prototype for use as an optimizing decision, 

that is, one in which the benefits to be achieved are balanced against a 

variety of costs. We will provide the information necessary to make this kind 

of a decision.

Importance of the Empirical Approach 
to the Development of the Prototypes

Many individuals have ideas about what characteristics an assessment 

system should have. We are committing ourselves to producing an assessment



2 3a

system which has two characteristics: it will be non-ciiscrimnatorv and job

relevant. Any other characteristics that it: may have will be determined by 

empirical analysis.

We are not committing ourselves in advance to any form of testing, though we 

suspect that a system to have validity will probably have a major component of per 

formance testing. We will provide exemplars of systems likely to have high valid 

for selecting the most competent individuals whatever form that system may take.

Work Schedule

A nine month time line is projected for this project. Work on the project 

will begin when the contract is signed and registered. Since this date is 

presently indeterminate, the schedule below describes the work plan in monthly 

modules.

1st Month:

1. Selection of the districts and schools in which the job analyses will 

be conducted and whose Boards and Superintendents will be interviewed.

2. Organization of the job analysis plan

3. Organization of the interview schedule to be used with Community 

Boards and Superintendents.

4. Organisation of job analysis and interviewing teams.

5. Selection of Advisory Board.

6 . First meeting of the Advisory Board.

7. Development of criteria for evaluating incumbents in the position.

2nd -_3rd Months :

1. Conduct of job analyses.

2. Conduct of interviews with Community Boards and Superintendents.

3. Preparation of job descriptions.

4. Preparation of specifications for testing procedures.

5. Rating of incumbents in the position.



24a

At.h Month:

1. Review of job descriptions ' v Community Boards and Superintendents.

2. Review of job descriptions and test speoificat ions by Advisory Board.

3. Development of testing procedures.

5th - 6th Months:

1 . Development of testing proce dures.

1
2 . Review of testing procedures b v Co mm un i t v Bo a r d s and Superintendents.

t
3. Review of testing procedures by Advisory Boards.

4. Pilot testing of instruments •
7th - 8th Month.:

1 . Analysis by Communitv Boards and Superintendents of their information

needs in selecting potential candidates.

2. Development of information system to be used in conjunction with the 

assessment system.

3. Data-gathering for validity studies.

4. Analysis of data for validity studies.

9th Month:

1. Draft of final report.

2. Review of draft of final report by Advisory Board.

3. Preparation of final report.

The final report of this project will be presented within two weeks

after the end of the ninth month.

Personnel

Vitae of project personnel who are currently on ETS's staff and who 

will be working on this project are in Appendix C. The amount of time 

allocated to this project by these staff members is specified in the Project 

Budget. Additional staff such as research assistants will be hired or selected

from the ETS staff.



i