Pegues v. Mississippi State Employment Service Brief for Plaintiffs-Appellants
Public Court Documents
July 21, 1981

Cite this item
-
Brief Collection, LDF Court Filings. Pegues v. Mississippi State Employment Service Brief for Plaintiffs-Appellants, 1981. ecb682fb-c09a-ee11-be36-6045bdeb8873. LDF Archives, Thurgood Marshall Institute. https://ldfrecollection.org/archives/archives-search/archives-item/f75ce94f-d4c2-4bee-904a-bfb805f04214/pegues-v-mississippi-state-employment-service-brief-for-plaintiffs-appellants. Accessed April 29, 2025.
Copied!
IN THE UNITED STATES COURT OF APPEALS FOR THE FIFTH CIRCUIT No. 80-3212 ROSIE LEE PEGUES, et al., Plaintiffs-Appellants, v. MISSISSIPPI STATE EMPLOYMENT SERVICE, UNITED STATES SECRETARY OF LABOR, and UNITED STATES EMPLOYMENT SERVICE, et al■, Defendants-Appellees. On Appeal from the United States District Court for the Northern District of Mississippi BRIEF FOR PLAINTIFFS-APPELLANTS WILLIAM L. ROBINSON RICHARD T. SEYMOUR LESTER GOLDNER Lawyers' Committee for Civil Rights Under Law 520 Woodward Building 733 Fifteenth Street, N.W. Washington, D. C. 20005 NAUSEAD STEWART Lawyers' Committee for Civil Rights Under Law 720 Milner Building 210 South Lamar Street Jackson, Mississippi 39201 DOROTHY A. WINSTON 824 Second Avenue, North Columbus, Mississippi 39701 Attorneys for Plaintiffs-Appellants Dated: July 2.1, 1981 Pegues v. Mississippi State Employment Service, No. 80-3212: CERTIFICATE OF INTERESTED PERSONS The undersigned, counsel of record, certifies that the fol lowing listed persons and agencies have an interest in the outcome of this case. These representations are made in order that the Judges of this Court may evaluate possible disqualification or recusal. Plaintiffs: and Plaintiffs-Intervenors: Rosie Lee Pegues Rebecca Gillespie Mary Boyd Robert Williams Percy Bell Sletta D. Brown Mary Hervey Christine Hodges Phillip Milan Pauline Willis Minority Peoples Council on the Tennessee-Tombigbee Waterway Defendants: Mississippi State Employment Service of the Mississippi Employment Security Commission Ernest Lindsey John Aldridge United States Secretary of Labor United States Employment Service Class Members: All black, and all female, past, oresent, and future applicants for employment referrals through the Bolivar County branch office of MSES (certified class) All black, and all female, past, present, and future applicants for employment referrals through MSES (proposed Statewide class for injunctive purposes only) RICHARD T. SEYMOUR Attorney of Record for Plaintiffs- Appellants STATEMENT CONCERNING ORAL ARGUMENT Plaintiffs-appellants request the Court to hold an oral argument in this case. This appeal comes to the Court after a three- week trial, in the course of which some hundreds of exhibits were received and numerous witnesses testified. The opinion below did not go into the facts in detail, and plaintiffs believe that oral argument would be of substantial assistance to the Court in coming to grips with the record. Moreover, the testing issues in this case are complex, and oral argument may well help clarify them for the Court, INDEX Page Table of Authorities vi Statement of Issues 1 Statement of the Case 2 A. Course of Proceedings and Disposition in the Court Below 2 B. Statement of the Facts 5 1. The Parties 5 2. The Importance of the Functions Performed by MSES 6 3. Stipulated Practices of MSES in Classifying and Referring Applicants 8 4. Evidence of Discrimination in Referrals 9 a) The Acceptance and Servicing of Job Orders with Stated Sex Preferences 9 b) Racially and Sexually Segregated Referrals on Job Orders 11 c) Selecting Applicants for Referrals on the Basis of Unvalidated Educational Requirements 15 d) Selecting Applicants for Referrals on the Basis of Unvalidated Experience Requirements 18 e) Comparison of Classifications and Referrals in 1970 19 f) Sharp Increases in the Rates of Black Referrals After the Filing of Suit 21 g) Sharp Increases in the Rates of Black Referrals to Material Handler Jobs, After the Cessation of Business with Travenol Laboratories Made it a Lower-Paid Job Area 23 h) Racial and Sexual Differences in the Rates of Pay for Jobs to Which Referrals Were Given, 1970-1978 24 i) Racial and Sexual Differences in Rates of Referrals 27 j) Referrals Out-of Code 27 -i- k) Other Classwide Evidence of Discrimination in Referrals 28 1) Dr. Malone's Analysis 30 m) The Defense that Job Openings Were Scarce 34 5. Evidence of Discrimination in the Classification of Applicants 35 a) The Relationship Between Classification and Referral 35 b) The Process by Which Occupational Codes Are Assigned 36 c) Classifications and Referrals in Service and Farmwork Occupations 40 d) Sharp Increases in the Numbers of Black Women Assigned to Clerical and Sales Codes After the Filing of Suit 43 e) Racial and Sexual Differences in the Class ification of Female Applicants with a Seventh- Grade or Lower Level of Education, 1974 44 f) Other Evidence of Discrimination in Class ification 44 6. Discrimination in Testing 47 a) Evidence of Disproportionately Adverse Impact of the Challenged Tests on Blacks 48 b) Evidence as to Validation 53 1) Evidence Other Than Dr. Hunter's Testimony 53 2) Dr. Hunter's Testimony 58 7. The Named Plaintiffs and Class Member Witnesses 63 8. Evidence on Class Determination 65 Summary of Argument 66 Argument 67 A. Plaintiffs Established a Prima Facie Case of Discrimination Which Has Not Been Rebutted 67 -11- 72 B. Plaintiffs Have Shown That the Challenged Tests Had a Racially Disparate Inpact and the Defendants Have Not Shown That They Were Valid C. The District Erred in Granting Summary Judgement for the Federal Defendants 75 D. The District Court Erred in Failing to Certify a State wide Class 77 Conclusion 78 -in- 8 12 13 21 22 23 23 24 25 26 26 33 32 34 LIST OF TABLES Percentage of Non-agriculture Referrals Resulting in Hire, as Shown on the State Defendants' Self- Appraisal Forms Referrals to Travenol on Material Handler and Assembles Job Orders (Plaintiffs' Exhibit 1, Volume III) Standard-Deviation Analysis of Referrals of Women on Material Handler and Assembler Job Orders frcm December 1969 through May 1970 (Travenol laboratories) Rates of Classification and of Referral, by Race and Sex in 1970 Standard Deviation Analysis of Rates of Classification and of Referral in 1970 Proportions of Referrals Given to Blacks, 1970-1973 Proportions of Male Referrals to Structural Work Jobs Other than Construction labor, and to Construction labor Jobs, Given to Blacks, 1970-1973 Proportions of Referrals to Material Handler Jobs Given to Blacks, 1970-1973 Average Hourly Pay Rates Per Referral Given, 1970-1973 Average Hourly Pay Rates Broken Down by Applicants' Level of Education, per Referral Given, 1975-1976 Average Hourly Pay Rates, Broken Down by Status as Veteran or Nonveteran, per Male Referral Given, 1975-1976 Percent of Job Orders Falling Within Dr. Malone's Group 2 or Group 3 (Questionable Probability or High Probability That More Referrals to the Job Order in Question Should Have Been Given) No. of Job Orders Closed. Without Referrals, or Closed With One Referral Numbers of Referrals Made on Job Orders, 1970-1972 -IV- 41 Table 15 Referrals of Black Women to Service Occupations, 1970-1973 41 Table 16 Results of Efforts to Upgrade the Codes of Male Applicants With an 8th-Grade or Better Level of Education, Classified in Service or Farmwork Codes, 1970-1976 43 Table 17 Increases in the Numbers of Black Women Assigned to Clerical and Sales Codes, FY 1972 to FY 1974 43 Table 18 Standard-Deviation Analysis of the Referrals to the Nurse's Aide Training Program at East Bolivar County Hospital 48 Table 19 Test Results fot the Licensed Practical Nurse SATB Reported on Application Forms 51 Table 20 Standard-Deviation Analysis of the Test Results for the Licensed Pratical Nurse SATB Reported on Application Forms 52 -v- TABLE OF AUTHORITIES A. Cases Pages Albemarle Paper Co. v. Moody, 422 U.S. 405 (1975) 74,75 Baxter v. Savannah Sugar Refining Corp., 495 F.2d. 437 (5th Cir.), cert. den. 419 U.S. 1033 (1974) 71 Boston Chapter NAACP v. Beecher, 504 F.2d 1017 (1st Cir., 1974), cert, den., 421 U.S. 910 (1975) 74-75 Burns v. Thiokol Chemical Corp., 483 F .2d. 300 (5th Cir., 1973) 74 Corley v. Jackson Police Dept., 566 F .2d. 994 (5th Cir., 1978) 68 Davis v. Califano, 613 F .2d. 957 (D.C. Cir., 1979) 69 Diaz v. Pan American World Airways, 442 F.2d. 385 (5th Cir.,), cert, den., 404 U.S. 950 (1971) 69 Dothard v. Rawlinson, 433 U.S. 321 (1977) 69,72 EEOC v. United Virginia Bank, 615 F-2d 147 (4th Cir., 1980) 68 Ensley Branch of NAACP v. Seibels, 616 F.2d. 812 (5th Cir.), cert, den., 66 L.Ed.2d 603 (1980) 73,75 Falcon v. General Telephone Co. of the Southwest, 626 F.2d. 369 (5th Cir., 1980), vacated and remanded on other issue, 49 U.S. Law Week 3743 (1981), opinion reinstated in relevant part, 647 F.2d. 663 (5th Cir., 1981) 69 Geller v. Markham, 635 F .2d. 1027 (2nd Cir., 1980) 73 Grant v. Bethlehem Steel Corp., 635 F.2d 1007 (2nd Cir., 1980), cert, den., 49 U.S. Law Week 3926 (1981) 69 - vx- Pages Griggs v. Duke Power Co., 401 U.S. 424 (1971) 70 Hameed v. Int'l Ass'n of Bridge Workers, 637 F .2d 506 (8th Cir., 1980) 68 Interstate Circuit v. United States, 306 U.S. 208 (1939) 73 Int'l Bhd. of Teamsters v. United States, 431 U.S. 324 (1977) 68,71,72 James v. Stockham Valves & Fittings Co., 559 F.2d. 310 (5th Cir., 1977) , eert. den., 434 U.S. 1034 (1978) 69,71,72,75 Johnson v. Goodyear Tire & Rubber Co., 491 F .2d. 1364 (5th Cir., 1974) 70 McDonnell Douglas Corp. v. Green, 411 U.S. 792 (1973) 71 Parson v. Kaiser Aluminum & Chemical Corp., 575 F.2d. 1374 (5th Cir., 1978), cert. den., 441 U.S. 968 (1979) 70,71 Payne v. Travenol Laboratories, 565 F.2d. 895 (5th Cir.), cert. den.. 439 U.S. 835 (1978) 77 Phillips v. Joint Legislative Committee, 637 F.2d. 1014 (5th Cir., 1981) 69 Rogers v. Int'l Paper Co., 510 F.2d. 1340 (8th Cir.), vacated on other grounds, 423 U.S. 809 (1975), modified in other respects, 526 F.2d. 722 (8th Cir., 1975) 74,75 Rowe v. General Motors Corp., 457 F .2d. 348 (5th Cir., 1972) 68,75 Shelak v. White Motor Co., 581 F .2d. 1155 (5th Cir., 1978) 73,74 Sledge v. J.P. Stevens & Co., 585 F .2d 625 (4th Cir., 1978), cert. den., 440 U.S. 981 (1979) 70,72 Swint v. Pullman-Standard, 539 F .2d. 77 (5th Cir., 1976) 69 Pages Texas Dept, of Community Affairs v. Burdine, U.S. , 67 L .Ed.2d 207 (1981) 71 United States v. City of Chicago, 59-9 F.3d. 415 (7th Cir.), cert, den., 434 U.S. 875 (1977) 74,75 United States v. County of Fairfax, 629 F .2d. 932 (4th Cir., 1980) 69 United States v. Georgia Power Co., 474 F .2d. 906 (5th Cir., 1973) 70,74 United States v. Jacksonville Terminal Co., 451 F .2d. 418 (5th Cir., 1971), cert, den., 406 U.S. 906 (1972) 74 Vulcan Society of N.Y.C. Fire Dept. v. Civil 490 F .2d. 387 (2nd Cir., 1973) Service Comm’n, 72 Vuyanioh v. Republic Nat'l Bank of Dallas, 505 F. Supp. 224 (N.D. Tex., 1980) 68,72 Ward v. Apprice, 6 Mod. 265 (Q.B., 1705) 73 Weeks v. Southern Bell Telephone & Telegraph 408 F.2d. 228 (5th Cir., 1969) Co., 69 B. Constitution. Statutes, Regulations and Rules Constitution, Fifth Amendment 4 Constitution, Thirteenth Amendment 4 Constitution, Fourteenth Amendment 4 Equal Employment Opportunity Act of 1972, Pub.L. 92-261, 86 Stat. 103 4,75,76,77 Title VII of the Civil Rights Act of 1964, 42 U.S.C. §§ 2000e et seq. passim §701 (c) of the Civil Rights Act of 1964, 78 Stat. 253-54 75,76 42 U.S.C. §1981 4 42 U.S.C. §1983 4 EEOC Guidelines, 29 C.F.R. §§ 1607.1 et seq. (1972), reprinted in relevant part at 2a 75 -viii- Pages Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. §§ 1607.1 et seq. (1980) reprinted in relevant part at Rule 52(a), F.R.Civ.P. C. Other Authorities American Psychological Association, Standards for Educational & Psychological Tests (1971+) 2 Conrad, Modern Trial Evidence §960 (1956) Senate Subcommittee on Labor of the Committee on Labor and Public Welfare, "Proposed Equal Employment Opportunities Enforcement Act of 1971, S.2515, S.2617, and H.R.17M-6, Bill Texts, Section by Section Analyses, Changes in Existing Law, Comparison of Bills Introduced" (1971) (Reprinted in Legislative History of the Equal Employment Opportunity Act of 1972 (1972)) 2 Wigmore, A Treatise on the Anglo-American System of Evidence §291 (3rd ed., 191+0) 67 ,73,7i+,75 68 57,74 73 IN THE UNITED STATES COURT OF APPEALS FOR THE FIFTH CIRCUIT No. 80-3212 ROSIE LEE PEGUES, et al., Plaintiffs-Appellants, v. MISSISSIPPI STATE EMPLOYMENT SERVICE, UNITED STATES SECRETARY OF LABOR, and UNITED STATES EMPLOYMENT SERVICE, et al. Defendants-Appellees. On Appeal from the United States District Court for the Northern District of Mississippi BRIEF FOR PLAINTIFFS-APPELLANTS STATEMENT OF ISSUES 1. Did the district court err in finding that the State and the Federal defendants had not discriminated against black, and against female, applicants for employment referrals at the Bolivar County branch office of the Mississippi State Employment Service ("MSES") and at other branch offices across the State? 2. Did the district court err in allowing the Federal defendants to present surprise expert testimony on test validation at the trial, where the expert propounded novel theories on which the Court relied, and where the effect of this surprise manuever was to deprive plaintiffs of any effective opportunity either to discover any problems which may have existed in the research on which he based his theories, or to prepare for effective cross- examination? 3. Did the district court err in holding that the 1972 amendments to Title VII of the Civil Rights Act of 1964- withdrew that Act's coverage of the United States Employment Service? <4. Did the district court err (a) in denying the applications of a number of blacks, harmed by the identical practices of MSES in other counties, to intervene as plaintiffs in this case, and (b) in denying plaintiffs' motion to certify a Statewide class for the purposes of injunctive relief? STATEMENT OF THE CASE A. Course of Proceedings and Disposition in the Court Below The plaintiffs filed their original EEOC charges in March 1970, and timely filed the Complaint after receipt of Notices of Right to Sue. They filed supplemental charges in 1973 and 1974, and timely moved to amend the Complaint after receipt of additional Notices. The motions were granted. 1/ (Stipulation, 59 2-5; stipulation exhibits 2-6, 10-13, 16-19, 21-24; R. 1, 2/ 862, 866, 1497, 1609. The original and amended Complaints in this action alleged across-the- board classwide discrimination against blacks and against women who applied to MSES for referrals to employers with available job vacancies. (R. 4-6, 129-33) The original Complaint alleged a countywide class, and the Amended Complaint 1/ The stipulation and the stipulation exhibits were received in evidence at trial. (Tr. 45.) All "Tr." references are to the trial transcript. 2/ The original and Amended Complaints alleged both racial discrimina tion against blacks and sexual discrimination against women. (R. 4-6, 129-33.) One of the original plaintiffs, Willie Mae Payne, had alleged both race and sex discrimination in her 1970 EEOC charge. (R. 154). On January 10, 1974--- while the Order has a typed 1973 date, this is clearly In error--she was dropped as a named plaintiff. (R. 544) . The EEOC charges filed in January and March, 1974, by plaintiffs Pegues, Gillespie and Boyd alleged both race and sex discrimination. (Stipulation, 99 2-5; stipulation exhibits 4, 12 and 23) . -2- alleged a Statewide class. (R. 3, 128) . On April 3, 1974-, plaintiffs moved to certify such a Statewide class. (R. 633). On November 16, 1977, six unsuccess ful black applicants for employment referrals at the Aberdeen, Amory, Green ville, Picayune, Tupelo, and West Point offices of MSES, as well as the Minority Peoples Council on the Tenessee-Tombigbee Waterway, applied for leave to intervene as plaintiffs and additional class representatives, to strengthen class representation and to support certification of a Statewide class. (R. 1561-63). On February 6, 1978, the named plaintiffs modified their 197M- motion to seek certification of a Statewide class solely for purposes of obtaining a declaration on liability and for injunctive relief. (R. 1671). At a hearing held on February 27, 1978, the district court announced that it would not certify a Statewide class, but would grant Statewide injunctive relief if plaintiffs proved Statewide discrimination, arising from practices similar to those followed in the Bolivar County office. (February 27, 1978 Hearing Tr. 253-56, App. 93-96). The orders denying the applications for leave to intervene, and certifying a countywide class of blacks and of women seeking employment referrals at the Bolivar County branch office of MSES, were entered on March 8, 1979. (R. 1782A, App. 59; R. 1788, App. 61). The defendants are described in Stipulation 6-21, and the claims against them were set forth in the Amended Complaint (R. 125). Mr. Lindsey has been Manager of the Bolivar County branch office of MSES since November 1968, and Mr. Aldridge was Executive Director of the Mississippi Employment Security Commission, which operates MSES, from July 1960 until February 1978. The State defendants have asserted a cross-claim against the Secretary, alleging that any liability on their part was "was the result of policies the U.S. Secretary of 3/ The Minority Peoples Council alleged that it had 800 members in Mississippi alone, and that some of its members had claims against MSES. -3- Labor has required. MSES to follow." (R. 610; R. 2035, App. 70) . On March 28, 1978, the district court entered summary judgment on all claims raised by plaintiffs against the Federal defendants, holding that the 1972 amendments to Title VII withdrew the United States Employment Service from the reach of Title VII, that the United States was immune from suit under 42 U.S.C. §1981 and the Thirteenth Amendment, and that the pleadings and the record did not show intentional conduct in violation of the Fifth Amendment. The State defendants' cross-claim against the Secretary was allowed to stand. (R. 1791, App. 63) . On December 27, 1978, the district court entered the agreed Pretrial Order, to which witness lists were then attached. Paragraph 18 of the Order recited that it would control the course of trial, and that "it may not be amended except by consent of the parties and the Court, or by order of the Court to prevent manifest injustice." (R. 2034B, 204-2, App. 69, 77). Six days before the start of trial, the Federal defendants notified plaintiffs of their intention to call an expert witness, John Hunter, Ph.D., to testify on the test validation issues. (R. 2076). These issues had been present in this case since the filing of the original Complaint. (R. 5). Plaintiffs took a hurried deposition of Dr. Hunter on July 5 and 6, 1979. The Federal defendants moved on the first day of trial for leave to call Dr. Hunter, i+/ The original Complaint alleged violations of Title VII of the Civil Rights Act of 1964, 42 U.S.C. §§ 2000e et seq.,_and of 42 U.S.C. §§ 1981 and 1983 (R 2). The Amended Complaint alleged violations of these provisions, and 'also alleged violations of the Fifth, Thirteenth and Fourteenth Amendments. (R. 134, 137). Allegations of claims arising under other provisions have been dismissed, (R. 1240; R. 1791, App. 63), and are not involved on this appeal. 5/ Fourteen months before the entry of this ruling, plaintiffs had^ filed their Second Motion to Compel Discovery, seeking to obtain information bearing on this question (R. 1287). The district court denied the discovery motion a month before it entered summary judgment against plaintiffs because of their failure to produce such information. (R. 1790, App. 62) . 4/ -4- (R. 2075), and plaintiffs opposed the motion on the grounds that Dr. Hunter was being called to support a novel approach to test validation which was radi cally different from the work previously done in the field, that he was going to rely on 26 articles which he had published or which were in the process of being published, that plaintiffs had received only 16 of the 26 articles by the Friday evening before the start of trial, that the subject matter of Dr. Hunter’s testimony was very involved, and that it would be extremely difficult for plaintiffs’ counsel to be able to check his work and to conduct an effective cross-examination. (Tr. 14— 16). The Court granted the Motion. (Tr. 16-18; 6/ R. 2080, App. 78) . Trial was held from July 9 through July 26, 1979. The district court entered its decision on March 10, 1980, finding all issues in favor of the defendants. (Decision, R. 2110, App. 79) . Judgment was entered against plaintiffs on their claims against the defendants, and was entered against the MSES defendants on their third-party claim against the Federal defendants. (R. 2174, App. 137) . Plaintiffs filed their Notice of Appeal on March 14-, 1980. (R. 2175) . B. Statement of the Facts 1. The Parties The four named plaintiffs are black residents of Bolivar County who attempted to obtain employment through the services of the Bolivar County branch office of MSES. Three are female, and one is male. They did not seek employ ment with MSES, but sought referrals to employers with jobs available to be 6/ While plaintiffs were also allowed to call an additional expert, they did not do so because there was not sufficient time for such an expert to study Dr. Hunter's research and to prepare meaningful testimony. Thus, the research predicates of Dr. Hunter's opinion, on which the trial court placed heavy reliance, were never subjected to any verification. -5- filled by hire. (Stipulation, 59 2-5; stipulation exhibits, 1, 7-9, 14—15, and 20). Their claims are discussed at 63, infra. 2. The Importance of the Functions Performed by MSES The primary functions of MSES are to help job applicants to find employ ment, and to help employers with job openings to fill their jobs. (Stipula tion, 5 22) . One of the ways in which MSES is to serve its primary function is to inform qualified applicants of job opportunities they had not known existed, and thus may not have discovered on their own, and by referring them to such vacancies. (Stipulation, 5 23). Providing information about such job opportunities can be extremely important to black applicants, who may have no other practical means of finding out about job vacancies for many local 7/ employers. By agreement with local employers, MSES sometimes pre-screens all ap plicants for production jobs and decides which applicants to refer. In such cases, the employer will not consider any applicant who has not been referred by MSES. From February 1965 to November 1971, for example, the Baxter ("Travenol") plant in Cleveland refused to consider applicants for its major entry-level jobs-- Assembler and Material Handler-- unless those applicants had first been referred by the Cleveland branch office. (Testimony of Olin Taylor, the former Baxter personnel manager, Tr. 276-78; testimony of Mr. Lindsey, Tr. 980-81). Mr. Lindsey thought that similar arrangements may have existed with respect to two other large local manufacturers although he was somewhat uncertain in his testimony, but he was convinced that MSES pre-screened 7/ Testimony of Rebecca Gillespie, Tr. 65-66; testimony of Mary Boyd, Tr. 14-0-4-1. Both of these plaintiffs live in Mound Bayou. (Id.) . Mound Bayou is an all-black city, and most job opportunities are in other parts of the county. (Testimony of Calvin Jones, Tr. 1894-95, 1910). -6- all of the applicants for the Licensed Practical Nurse training program at East Bolivar County Hospital. (Tr. 993-95) . Where an employer will only con sider an applicant who has been referred by MSES, a refusal by MSES to refer an applicant will bar that applicant from consideration for hire. Referrals by MSES account for a substantial share of the hires by lar ger local employers. Mr. Lindsey prepared an analysis of hires by the 85 or 90 8/ local employers with ten or more employees. In the time periods covered by his analysis, the Bolivar County branch office was responsible for filling up to 43.9% of the new hires of these "major market" employers. From January 1969 through December 1970, a period which includes the filing of EEOC charges herein the Bolivar County branch office placed 1,030 of the 3,356 new hires of these "major market" employers, or 30,7% of the total. From January 1976 through August 1978, the Bolivar County branch office placed 732 of the 3,421 new hires of such employers, or 21.4% of the total. (Calculation of counsel from State defendants' exhibit 32) . While giving a referral to an employer with a job vacancy does not guarantee that the person referred will be hired, the vast majority of persons referred are in fact hired. This is true for all applicants in general, for black applicants, and for female applicants. State defendants’ exhibits 44 through 52, and 55, contain monthly year-to-date statistics on nonagricultural referrals and placements from 1973 through late 1978. They show the following: 8/ Testimony of Mr. Lindsey, Tr. 2226-27. Mr. Lindsey’s analysis did not include placements with, or new hires of, the several hundred local employ ers with fewer than ten employees. Tr. 2227-28. -7- Table 1. Percentage of Non-agricultural Referrals Resulting in Hire, as Shown on the State Defendants’ Self-Appraisal Forms_____ Percentage of Non-Agricultural State Referrals Resulting in Hire Defendants’ Date of All Black 9/ Female Exhibit No. Appraisal Form Applicants Applicants Applicants 55 11/9/73 93.1% 93.6% N.A. 94 7/5/74 75.2% 73.7% 70.3% 45 10/10/74 72.9% 70.7% 69.5% 46 6/26/75 71.4% 98.1% 69.8% 48 10/24/75 72.2% 64.7% 70.5% 49 6/8/76 76.2% 71.6% 73.2% 47 10/18/76 73.1% 68.5% 68.1% 50 6/10/77 67.5% 61.2% 77.8% 51 2/23/78 69.1% 63.8% 55.7% 52 8/16/78 67.7% 63.3% 61.8% Thus, the decision whether or not to give a referral to an applicant have a substantial effect on that applicant's chances of obtaining employment. 3. Stipulated Practices of MSES in Classifying and Referring Applicants The general operations of the Bolivar County branch office of MSES are set forth in some detail in Stipulation 59 22-61. In summary, applicants for employment referrals come to the office, are interviewed and occupational codes are assigned to them. The code is supposed to represent the type of work for which the applicant is best qualified. The actual practices of MSES in classi fying applicants are described at 35-47̂ infra. When a job order is received from an employer, the employer’s job is also assigned an occupational code. MSES then makes referrals by matching the occu pational codes of active applications with the occupational codes of active job orders, and by matching the education, experience, and other requirements set by the employer with the characteristics of the applicants being considered. The actual practices of MSES in selecting applicants for referral are discussed at 9-35, infra. 9/ Data for "Minorities" was used where data for blacks was not stated. -8- 4-. Evidence of Discrimination in Referrals a) The Acceptance and Servicing of Job Orders with Stated Sex Preferences Plaintiffs' exhibit 57 is a list of job orders, accepted and serviced by the Bolivar County branch office of MSES from November 1969 through June 1973, which contained specifications that only men, or only women, be referred on the job order. (Testimony of Linda Thome, Tr. 635-39) . The exhibit shows that MSES honored the employers’ sexual preferences, referring only males on job orders specifying an employer preference for males, and referring only females on job orders specifying an employer preference for females. The U.S. Department of Labor had cautioned all State Employment Services in 1965, 1966, 1967, and 1970 that accepting and servicing such sex specifica tions on job orders was unlawful under Title VII except where sex Is ”a bona fide occupational qualification reasonably necessary to the normal operation of that particular business or enterprise." (Stipulation 5 74, stipulation exhibits 51, 52; Federal defendants’ exhibits 11, 17; State defendants’ exhibits 23, 26 (both documents)) . Stipulation exhibit 52 expressly cautioned State Employment Services that they will "share responsibility with the employer" if they service such sex preferences without evidence that they are a BFOQ, and caution that a written record must be kept whenever a sex specification is accepted as a BFOQ. On June 29, 1968, the U.S. Department of Labor wrote to the defendant Aldridge, informing him that a review of the Cleveland (Bolivar County) and other branch offices of MSES had shown that all of the offices in vestigated were accepting and servicing discriminatory sex specifications on job orders. (Plaintiffs’ exhibit 114, p. 10, part VI). In his August 19, 1968 response, Mr. Aldridge did not quarrel with the Department’s findings as to the Cleveland office, and promised to institute a compliance review process. -9- (Plaintiffs’ exhibit 158 at p. 5). The problem was again brought to Mr. Aldridge’s attention in an interoffice memorandum dated May 5, 1969. (Plain tiffs’ exhibit 115, p. 3). Nevertheless, plaintiffs' exhibit 57 showed that the problem continued in the Cleveland office for more than four years after the date of this memorandum. When questioned about this practice, Mr. Aldridge stated that he thought "the sex part of the Act came along a little bit later", that "the Department of Labor and the others were not quite as quick to tackle it", and that the question was in "a transitory period for a while as to just what to do ... .’’ "And in the South we, some people just didn’t think that a woman ought to be in certain things." Tr. 1226. Although plaintiffs' exhibit 114- had put Mr. Aldridge on notice that the Cleveland office had engaged in this discriminatory practice and that it should not be continued, Tr. 1227-28, he stated that he "had no reason" to inquire thereafter whether the Cleveland office had stopped the practice, and was not aware that it had continued until 1973. (Tr. 1229-30). Mr. Lindsey also testified that he thought that "they didn’t start at the same time. I think the sex came along later.” (Tr. 956-57). He testified that he was unsure whether he had known in 1969 that the prohibition of sex discrimina tion was in effect, but he thought he did. (Tr. 961). He had no personal knowledge of the job orders listed on plaintiffs’ exhibit 57, and could not say whether there was any investigation into the question whether those sex pre ferences were justified as BFOQ’s; if there had been such an investigation and it was not recorded on the job order, he knew of no place where it could have been recorded. (Tr. 959-60). A Bulletin sent to MSES local office managers on September 8, 1969, required any such investigations to be recorded on the job order. (State defendants' exhibit 25, first document p. 2). The bulk of the job orders in question were included in plaintiffs’ exhibits 1, 2, and 3, and -10- none of them show any sign of such an investigation. Louis Beverly, Jr., the Equal Employment Opportunity officer for MSES (Tr. 2239), agreed on cross-examination that State Defendants' exhibits 23 and 26 should have alerted the staff of the Cleveland office in 1966 and 1967 that it was unlawful to engage in such practices. (R. 2289-91). At trial, Mr. Lindsey was asked whether his office now accepted the types of job orders shown on plaintiffs’ exhibit 57. He answered: "We do not. I think our improvement, if that is what you want to call it, has gotten to the point that we do not." (Tr. 963). The trial court’s decision did not mention the evidence as to this prac tice, but stated only that ”[l]ocal offices are prohibited from processing job orders that contain unlawful sex specifications." (R. 2142, App. 108). b) Racially and Sexually Segregated Referrals on Job Orders Mr. Lindsey testified at trial that Travenol Laboratories required applicants to its Cleveland plant to be referred by MSES before they would be considered by the company, that this continued from at least 1965 through November 1971, that he had never seen any sex specification on the master job order forms for the Assembler and Material Handler jobs, and that he did not remember any instructions from Travenol’s Personnel Manager as to the sex of the referrals to be made on these two different jobs. (Tr. 980-85) . Travenol’s former Personnel Manager, Mr. Taylor, also testified that he did not recall authorizing MSES to refer only males on the Material Handler job orders and 10/ There can be no contention that the defendants were surprised by the listing of job orders on plaintiffs’ exhibit 57, or that they had not had suf ficient time to inquire into those job orders. Pursuant to the pretrial pro cedure followed in the trial court, the parties exchanged exhibits prior to the Pretrial Conference in October 1978 (see R. 2034B, 2038-40, App. 69, 73-75), and Mr. Lindsey and Mr. Aldridge had thus had copies of this exhibit nine months before the start of trial. (Plaintiffs’ exhibits 1 through 117 were included in this exchange). 10/ -11- 11/ only females on the Assembler job orders. (Tr. 280, 285-86). Plaintiffs' exhibit 1, volume III, shows referrals to Travenol on Material Handler and Assembler job orders. The Material Handler job then paid $2.21 an hour to start, and the Assembler job then paid $1.97 an hour to start. (Id., pp. 31, 38) . Disregarding referrals for part-time positions, fid.. p. 46) , referrals marked "FWD" ("foreward") indicating the recordation of hires of persons previously referred, and using MSES's own counts of referrals when ever possible (e ,g,, p. 4) , it is clear that MSES referred only males on Materi al Handler job orders in the six months ending in May 1970, and referred only females on Assembler job orders in the same period of time: Table 2. Referrals to Travenol on Material Handler and Assembler Job Orders (Plaintiffs' Exhibit 1, Volume III)_________ Page No. in Referrals of Referrals of PX-1, Vol. Ill Date of Job Order Males Females Material Handlers: 14-15 12/1/69 39 0 16-17 1/2/70 41 0 22-23 2/2/70 21 0 29-30 3/2/70 24 0 -31-32 5/5/70 23 0 33 4/1/70 4 0 34-35 4/21/70 21 0 48 1/20/70 10 0 Total 183 0 Assemblers: 1-2 11/3/69 0 45 3-5 12/1/69 0 42 6-7 12/22/69 0 43 8-9 2/2/70 0 45 10-13 3/2/70 0 63 38-39 4/1/70 0 18 40-43 5/5/70 0 45 Total 0 301 11 / Stipulation exhibits 59 and 60 show that MSES used the occupational titles "Packager, Solutions and Syringes" and "Table Worker" to describe the lower-paid Assembler job, and the occupational titles "Material Coordinator” and "Fork-Lift Truck Operator" to describe the higher-paid Material Handler job. Mr. Lindsey agreed. (Tr. 986) . The exhibits show that neither of these job categories required any prior experience, and that on-the-job training would be provided in each. -12- (A table covering referrals for a shorter period of time was introduced as plaintiffs’ exhibit 156). 37.9% of the 485 referrals were male, and 62.1% were female. Using these proportions to indicate the availability of applicants of each sex for these entry-level production jobs, Table 3 shows that it would be extremely difficult to explain this result by chance: Table 3. Standard-Deviation Analysis of Referrals of Women on Material Handler and Assembler Job Orders from December 1969 through May 1970 (Travenol Laboratories)_____________________________ Material Handler Job Orders Assembler Job Orders Availability of Women in Total Group Referred for Both Jobs 62.1% 62.1% Sample Size (No. of Referrals) 183 301 Expected No. of Female Referrals 114 187 Observed No. of Female Referrals 0 301 Difference - 114 + 114 Standard Deviation 6.6 • 00 No. of Standard Deviations Between Expected and Observed Nos. of Female Referrals -17.3 +13.6 Mr. Lindsey was shown the job orders in plaintiffs’ exhibit 1, volume III, and was asked to explain the sexually segregated referrals when the employ er had not given MSES a sexual preference to follow. He said that he could not explain it, that he thought there were sexually integrated referrals on later job orders, and that he did not know what brought about that change. (Tr. 986- 12/ 88, 992). 12/ One month after the last of the job orders reflected in Table 2, the MSES defendants were served with copies of the plaintiffs’ EEOC charges. (Plaintiffs’ exhibits 97, p. 1, and 101, p. 1). Some of the post-service-of- charges job orders in plaintiffs’ exhibit 2, volume III, show both males and females referred for the Assembler job (Id,, pp. 10 (a)-13 (c) , 15(a)-15(f), 17(a) 17 (i)) , although only males were referred on Material Handler job orders. (Id., pp. 7 (a)-7 (c) , 19). MSES staff at State headquarters were aware of these patterns of segregated referrals. (Plaintiffs’ exhibit 116). -13- A manual analysis of MSES referral records for the Bolivar County branch office for the period from November 1969 through December 1970, the period in which the initial EEOC charges in this case were filed, show that there were 185 job orders on which two or more applicants were referred. Of these, 175 job orders__ 94.6% of the total---resulted in either all-male or all-female re ferrals, and H O -- 59.5% of the total---resulted in either all-white or all black referrals. (Plaintiffs1 exhibit 58, p. 1; testimony of Linda Thome, Tr. 639-M-l, 681). Mr. Lindsey was asked for an explanation, but said that he 13/ could give no answer. (Tr. 992-93) . The trial court did not address this evidence in its decision. Instead, the court found nondiscrimination in reliance on Mr. Lindsey's testimony, "based on random examination of referrals on job orders", that referrals "were made without regard to the race or sex of the applicants referred", and on the basis of its view that MSES performed nondiscriminatorily, within the limita tions under which it labored. (R. 2123, 2125-26, App. 91-93). The "random examinations" of job orders mentioned by the court below were reflected in the "self-appraisal" forms introduced as State defendants' exhibits 14/ 44-55. Typically, the self-appraisal forms in evidence recite that a "random sample" of ten or so job orders was pulled from the file and found to be non- discriminatory. However, Mr. Lindsey testified that he did not pull any 13/ This was not an isolated phenomenon. Calculations of counsel from the 1972 job orders in evidence as plaintiffs' exhibit 3 show that there were 427 job orders with two or more referrals. Sex was identified for all refer rals on 424 job orders. Of these, 402 or 94.8% of the total resulted in referrals only of males, or only of females. There were 407 job orders with race identified for all referrals. Of these, 214 job orders 52.6% of the total resulted in referrals only of whites, or only of minorities. 14/ There was no self-appraisal form for 1970 or for 1971, and the district court's rationale is therefore inapplicable to the situation at the time the charges of discrimination were filed, or to the exhibits and testimony concerning the Travenol job orders. -14- "random samples" from the files; he asked the interviewers whose conduct he was examining to pull a "random sample" of the job orders they had handled. (Tr. 2233-34-) . He admitted that he gave them no instructions to follow in picking job orders for his review, because he believed that that would destroy the randomness of the sample. (Tr. 24-58-59) . He further admitted that the interviewers pulling out job orders for his review were probably aware that it was for the purpose of filling out a civil rights self-appraisal form. (Id.) The lower court sustained an objection to the question whether Mr. Lindsey actually knew if the job orders he reviewed were being pulled by interviewers on a random basis (Tr. 24-59) . Counsel for plaintiffs then asked the following question: Q. Did you ever take any steps to find out whether the interviewers in fact had pulled out the best possible job orders in terms of minority referrals? The MSES defendants objected on the basis that this constituted "harassment of the witness." The lower court sustained the objection. (Tr. 24-60). Subsequently, Mr. Lindsey admitted, as to the format of the self-ap praisal reports: "It just sort of lends itself to showing a satisfactory 15/ situation, just on the face of it.” (Tr. 24-78) . c) Selecting Applicants for Referral on the Basis of Unvalidated Educational Requirements ___________ _______________________ Mr. Lindsey testified that it was the policy of his office to accept whatever an employer sets down as its educational requirement, and to refer only applicants meeting the requirement, unless there were too few applicants 15/ Compare, for example, State defendants' exhibit 53, p. 2, the self- appraisal form dated August 30, 1972, which says "All orders for the six-month evaluation period were checked and no discriminatory specifications found as to age and sex", with plaintiffs’ exhibit 57, p. 3. -15- with that level of education. If that occurred, his office would then try to obtain some leeway from the employer with respect to the requirement. He could not recall any specific instance in which his office had in fact been short of people with the requisite level of education. (Tr. 976-78) . Lane Hart, the Director of MSES from November 1957 through August 1, 1975 (plaintiffs' exhibit 19-9, Dep.Tr. 66; plaintiffs’ exhibit 150, Dep.Tr. 5), and Charles Ballard, chief of the Programs and Methods Branch of MSES from 1956 through 1976 and thereafter Assistant Director of MSES, (Tr. 2520-21) , testified that efforts to negotiate an employer's educational requirement downward, or to seek waivers of the requirement on behalf of particular applicants, were not made when the local office had enough applicants with the requisite level of educa tion available for referral. (Plaintiffs’ exhibit 150, Dep.Tr. 24-25). MSES does not require the local office manager to obtain any facts from the employer indicating that the educational requirement can be validated. (Id., Dep.Tr. 29). Plaintiffs' exhibits 5 and 56 show a number of job orders with educa tional requirements serviced by the Cleveland branch office in 1970 and in 1976. Many of them are for ordinary clerical positions, or positions as waitress, cook, sales clerk, and so forth. Mr. Lindsey testified that his office had accepted basically the same types of educational requirements, and has serviced them, throughout the period of time he has been Manager of the local office. (Tr. 974-76). Census statistics show that educational requirements in Bolivar County and in the State of Mississippi have a disproportionately adverse effect on blacks. (Stipulation ?*[ 68, 69; stipulation exhibits 47, 48). Dr. Linda Malone, a statistician called by the State defendants, testified that black applicants in the Bolivar County branch office had a lower level of education than white applicants, that female applicants had a lower level of education than male -16- applicants, and that the differences were statistically significant. (Tr. 1429- 36; State defendants' exhibit 71). Plaintiffs' exhibit 8 shows that the propor tions of blacks among the applicants referred declines sharply as the level of the educational requirement or preference increases. 16/ There is no evidence of any business necessity for this practice. Even in the absence of an educational requirement set by an employer, MSES interviewers may decide to select one applicant for referral, instead of another, based in part upon the applicant's level of education. (Testimony of Mr. Lindsey, Tr. 1022-23). The district court found that acceptance by the MSES defendants of Travenol Laboratories' tenth-grade requirement discriminated against black appli cants, but that Travenol's use of MSES had ended in November 1971, prior to the judicial Complaint, and that plaintiffs and their class were accordingly not 17/ entitled to any relief on this issue. (R. 2168-69, App. 130-31). The court did not discuss the large number of similar job orders accepted and serviced by the MSES defendants after the filing of the judicial Complaint, but observed that regulations prohibited the local office from accepting employer requirements which result in the exclusion of applicants of a particular race. (R. 2140, App. 106) . 16/ Mr. Lindsey testified that employers would be generally receptive to some extent, if he called them and said that he did not have enough applicants meeting their educational requirements and asked permission to refer persons with a lesser level of education. He did not know whether employers would be any less receptive if he informed them that MSES would like to refer applicants whom MSES thought capable of doing the job, without regard to educational require ments. (Tr. 978-79). y 17/ Plaintiffs had filed their EEOC charges in March 1970, and the EEOC had also investigated the matter in 1970. (Plaintiffs' exhibits 96, 97, 101, 103) -17- d) Selecting Applicants for Referral on the Basis of Unvalidated Experience Requirements______________________________________ The evidence at trial established that MSES policies with respect to the servicing of experience requirements set by employers were identical to its policies with respect to the servicing of educational requirements set by em ployers. (Testimony of Mr. Lindsey, Tr. 979-80; testimony of Mr. Ballard, plaintiffs’ exhibit 150, Dep. Tr. 59-60; Stipulation 5 70); plaintiffs' exhibit 5) . The evidence at trial showed that there are substantial racial and sex ual differences in the types of previous employment experience possessed by applicants. Mr. Lindsey testified that, at least through the mid-1960's, the traditionally-black jobs in Bolivar County for males were service station attendant, cook, and general labor. The traditionally-black jobs for females were maids, cooks, restaurant cooks, char workers in motels, and industrial maids. The traditionally-white jobs were everything else. (Tr. 822-23). He further testified that, for the entire period of time covered by State defend ants' exhibits 49— 55-- the self-appraisal forms from 1972 through 1978-- blacks in Bolivar County had had less experience and training than whites. (Tr. 2465- 66). Mr. Jones, the Superintendent of the part of the local school district in Mound Bayou, testified that the background of the black population in Mound Bayou had essentially been farm labor, that the mechanization of farms had thrown them out of work, that they did not have the experience or skills for other work, and that, while some progress had been made, the continuing high rate of black unemployment indicated that the problem was still present. (Tr. 1894-98, 1908-09, 1918-19). Mr. Beverly testified that he had become familiar with the fact that a previous-experience requirement for a relatively high- paying job in the Mississippi Delta could have a disproportionately adverse -18- effect on blacks, and that for this reason he had included such situations in the civil rights training manual for local MSES staff. (Tr. 2281; State de fendants' exhibit 43 at pp. 44-46). Stipulation 72 and 73, and Stipulation exhibit 50, show 1970 Census statistics indicating substantial differences be tween the occupations held by blacks and those held by whites, and between those held by women and those held by men. The defendants did not introduce any evidence of business necessity for this practice. Mr. Lindsey testified that he had no basis for knowing whether employers would be unreceptive if his office informed them that it would refer applicants MSES thought could do the job, without regard to prior experience. (Tr. 980) . Even where employers have not specified an experience requirement, MSES relies heavily on the previous employment experience of applicants in deciding which applicants to refer to the employer. (Testimony of Mr. Lindsey, Tr. 1022; testimony of George Nash, Tr. 2077) . The district court did not address this issue in its decision, except by noting that employment service offices were prohibited from processing facially neutral orders with racially exclusionary effects. (R. 2140, App. 106). e) Comparison of Classifications and Referrals in 1970 Under MSES policy, an applicant with an occupational code identical or similar to the occupational code of a job order should be referred on the job order before other applicants are referred. (Stipulation, 15 38, 57-59). Mr. Lindsey testified that there was quite a bit of leeway within each code "family" or group of jobs with the same initial digit; if a job order had the initial digit 1', indicating a professional, technical, or managerial job, MSES would probably refer only persons whose occupational codes also began with "1". Ex ceptions were sometimes made, but this was the general rule. (Tr. 808-09; sti -19- pulation exhibit 36, vol. II, p. 1). The MSES defendants have sought to estab lish that the classificaticrB of applicants into various occupational codes reflect the qualifications, abilities, and interests of applicants, (Stipula tion, 39; testimony of Mr. Lindsey, Tr. 2166-67; decision, R. 2135-39, App. 102-05). Putting aside for the moment plaintiffs’ contention that there has been discrimination in the discharge of this function, a comparison of the rates at which blacks have been classified into particular code "families" with the rates at which blacks have been referred on job orders for the same "families" should show whether MSES has made racial distinctions in the referrals of applicants it had previously decided to be similarly qualified. Plaintiffs’ exhibit 80 shows the classification of applicants in 1970, and plaintiffs' exhibit 80 shows the classification of applicants in 1970, and plaintiffs' exhibits 61 and 62 show their referrals in 1970, by occupational codes. Table M- shows that there are substantial differences between the classification rates and the referral rates of black applicants to various code "families". An asterisk indicates that the difference is statistically significant, as indicated in table 5 on page 22. -20- Table 4. Rates of Classification and of Referral, by Race and Sex, in 1970 __________________________________________ Mean Black Males, as Per- Black Females, as Per- Hourly centage of All Males centage of All Females Code Description Wage Coded Referred Coded Referred 0-9 All Jobs — 57.2% 38.0%* 61.0% 48.7%* 0,1 Professional, Technical & Mgr. Jobs $1.96 34.1% 7.7%* 38.5% 0%* 2 Clerical and Sales Jobs $1.95 37.2% 19.8%* 32.5% 18.5%* 3 Service Jobs $0.93 82.9% 77.3% 83.7% 87.0% 1+ Farmwork Jobs $1.37 79.5% 58.6%* 94.1% 100 % 5 Processing Jobs $1.56 60.0% 42.9% 37.5% — 6 Machine Trades Jobs $1.72 52.5% 12.5%* 25.0% 100 % 7 Bench Work Jobs $1.59 38.7% 28.6% 35.1% 66.7%* 8 Structural Work Jobs $1.77 62.7% 55.1% 33.3% — 9 Miscellaneous Jobs $1.84 58.6% 65.9% .65.3% 23.6%* f) Sharp Increases Filing of Suit in the Rates of Black Referrals After the Plaintiffs introduced manual analyses of referrals to various types of occupations, broken down by race and sex. (Plaintiffs1 exhibits 61, 62, 65, 66, 69, 70, 73, and 74). The proportions of all referrals given to blacks in creased after service of the EEOC charges in mid-1970, and increased substan tially again in the year suit was filed; the proportions of referrals to other traditionally-white jobs went up markedly after suit. Table 6 on p. 23 shows the details. -21- -22 - Table 5. Standard Deviation Analysis of Rates of Classification and of Referral in 1970 _______ Males All Codes Prof., Tech. & Mgr. Codes Clerical & Sales Codes Farmwork Codes Mach. Trades Codes Availability: Blacks as Pro portion of Applicants With This Code .572 .341 .372 .795 .525 Sample Size: No. of Referrals to Jobs With This Code 505 13 253 29 8 Expected No. of Black Referrals 288.9 4.4 94.1 23.1 4.2 Observed No. of Black Referrals 192 1 50 17 1 -- Difference -96.9 -3.4 -44.1 -6.1 -3.2 Standard Deviation 11.1 1.7 7.7 2.2 1.4 No. of Std. Dev1ns Between Ex pected and Observed Values -8.7 -2.0 -5.7 -2.8 -2.3 Females All Codes Prof., Tech. & Mgr. Codes Clerical & Sales Codes Bench Work Miscellaneous Codes Codes Availability: Blacks as Pro portion of Applicants With This Code .61 .385 -325 .351 .653 Sample Size: No. of Referrals to Jobs With This Code 759 9 124 48 297 Expected No. of Black Referrals M-63.0 3.5 40.3 16.8 193.9 Observed No. of Black Referrals 370 0 23 32 70 --Difference -93.0 -3.5 -17.3 +15.2 -123.9 Standard Deviation 13.4 1.5 5.2 3.3 8.2 No. of Std. Dev’ns Between Ex pected and Observed Values -6.9 -2.3 -3.3 +4.6 -15.1 Table 6. Proportions of Referrals Given to Blacks, 1970-73 Type of Job % Black in 1970 % Black in 1971 % Black in 1972 % Black in 1973 All Jobs - Males 38.0% 59.2% 67.3% 78.7% - Females 48.7% 52.7% 68.9% 64.2% Clerical and - Males Sales 19.8% 25.5% 34.3% 52.7% - Females 18.5% 10.6% 28.8% 44.7% Professional, - Males Technical, Managerial 7.7% 32.3% 28.6% 58.4% - Females 0% 36.8% 27.8% 67.6% Plaintiffs’ exhibit 76 shows that the qualifications of many of the whites re ferred to Professional, Technical and Managerial jobs in 1970 were minimal. Until suit was filed, black male applicants received a disproportionately small number of referrals to structural work occupations other than construction labor (codes 80-85 and 87-89) , and a disproportionately large number of re ferrals to the lower-paid construction labor occupations (code 86): Table 7. Proportions of Male Referrals to Structural Work Jobs Other than Construction Labor, and to Construction Labor Jobs, Given to Blacks, 1970-1973_______________ Year Range of Pay Rates_____ Other Than Construction Construction Labor Labor __ Male Referrals To Structural Jobs Other Than Construction Labor -- % Black________ Male Referrals To Construction Labor Jobs --% Black_____ 1970 $1,94-$2.50 $1.60 1971 $1.71-$2.35 $1.69 1972 $1.87-$3.28 $1.82 1973 $1.68-$2.43 $1.77 32.6% 50.8% 75.4% 70.4% 87.5% 77.8% 78.9% 76.3% g) Sharp Increases in the Rates of Black Referrals to Material Handler Jobs, After the Cessation of Business with Travenol Laboratories Made it a Lower-Paid Job Area_________________ The Material Handler job at Travenol was much more highly-paid than other Material Handler jobs in the area which, from an examination of job orders 18/ seem to have involved manual labor loading and unloading trucks. In 18/ See, e,g., plaintiffs’ exhibit 1, vol. II, pp. 260, 264, 266, 268 and 272. -23- November 1971, Baxter Laboratories stopped, using MSES as its sole source of referrals, and the average hourly pay rate on Material Handler job orders declined accordingly. Plaintiffs' exhibits 61, 62, 65, 66, 69, 70, 73 and 74- show that, as the average pay rate of the Material Handler job orders declined, MSES increased substantially the proportions of blacks it referred on such job orders: Table 8. Proportions of Referrals to Material Handler Jobs Given to Blacks, 1 9 7 0 - 1 9 7 3 ____________________ Year Overall Average Hourly Pay Rate Male Referrals To Material Handler Jobs -- % Black Female Referrals To Material Handler Jobs -- % Black 1970 $1.89 60.4% 23.6% 1971 $2.06 55.3% 56.6% 1972 $1.79 84.5% 100% (only 1 woman) 1973 $1.57 84.1% 86.7% The district court did not address this evidence in its decision. h) Racial and Sexual Differences in the Rates of Pay for Jobs to Which Referrals Were Given, 1970-1978__________________ The U.S. Department of Labor has established a method of analyzing the activities of local employment service offices from the standpoint of civil rights compliance. Called "PEER” , it compares the average wage rates of the jobs to which applicants of each race and sex were referred, and compares the services provided to them with their representation in the population. Mr. Beverly testified that it was a very useful approach which might require him to take much harder looks at some things now than he would have earlier. (State defendants’ exhibit 80 at pp. 4-5; Tr. 2272-74, 2294— 95, 2312). From 1970 through July 1978, there were strong racial and sexual dis parities in the pay rates of jobs to which referrals were given: -24- Table 9. Average Hourly Pay Rates Per Referral Given, 1970-1978 White Male Black Male White Female Black Female Time Period Referrals Referrals Referrals Referrals 1970 $2.05 $1.67 ’$1780 $1.18 1971 1.98 1.78 1.96 1.70 1972 1.87 1.80 1.65 1.10 1973 1.96 1.73 1.70 1.42 1/74 to 6/74 2.26 2.13 1.86 1.58 7/74 to 6/75 2.36 2.19 2.11 2.09 7/75 to 6/76 2.64 2.41 2.32 2.27 10/76 to 9/77 2.80 2.57 2.56 2.40 10/77 to 7/78 3.05 2.77 2.78 2 .62 (Plaintiffs’ <exhibits 117, 119 at p. 7, 123 at p. 1) . Plaintiffs’ exhibit at p. 1 shows the levels of statistical significance for various comparisons for the period from 1974 to 1978; most of the differences are significant at 19/ the .0001 level. Plaintiffs’ exhibit 19(b) is a much more extensive analysis of referrals by each MSES office in the State for a period of approximately a year ending in 20/ March 1976. Although Dr. Malone had found statistically significant differ ences between the levels of education of blacks and whites and of males and females, pp. 369-76 of this exhibit show that any large overall racial or sexual differences remain even when comparing referrals of persons with the same levels of education: 19/ Because the statistics for the period 1970-1973 were calculated by hand rather than by computer, no calculations of significance were done for this period. 20/ Plaintiffs’ exhibit 19(b) at p. 369 shows that the tape reflects data for 5,785 separate applicants at the Bolivar County branch office, and plaintiffs’ exhibit 153 shows that there were 5,566 new or renewed applica tions filed in FY 1975. The MSES defendants supplied this tape to plaintiffs, bud did not inform plaintiffs of the data it covered. (Tr. 265-66) . Mr. Frodyma testified that data on the tape ended in March 1976. (Tr. 263-66). -25- Table 10- Average Hourly Pay Rates Broken Down by Applicants' Level of Education, per Referral Given, 1975-76 Educational Level of Applicants All Applicants 1-6 Years 7-9 Years 10-11 Years 12 Years 13-15 Years 16 or More Yea White Males $2.61 2.36 2.30 2.61 2.73 2.70 3 3.02 Black Males $2.39 2.31 2.37 2.38 2.37 2.53 2.92 White Females $2.26 2.10 2.12 2.15 2.27 2.31 2.46 Black Females $2.22 2.15 2.11 2.20 2.21 2.30 2.44 MSES is required to give veterans preference in referrals, 20 C.F.R § 653.221(a) (7) (1980), a provision primarily of benefit to men but not chal lenged in this case. Plaintiffs’ exhibit 19(b) at pp. 377-78 shows that white men who were not veterans fared substantially better than black men who were veterans: Table 11. Average Hourly Pay Rates, Broken Down By Status as Veteran or Nonveteran, per Male Referral Given, 1975-76 Status White Males Black Males Hourly Difference Annual Difference (2080 Hours) All Applicants $2.61 Veterans 2.72 Non-veterans 2,57 $2.39 22d $457.60 2.42 30d $624.00 2.39 18d $374.40 Mr. Nash was asked whether he could explain these racial disparities between black males and white males. He stated that he could not. (Tr. 2086- 87). The district court did not address this evidence in its findings, but its conclusions stated that "the human elements involved in affording a job appli cant an opportunity of gainful employment ... cannot be inserted into a compu ter" to show a true picture, and stated that plaintiffs’ charts and schedules did not reflect "all of the relevant and available information". (R. 2167, App. 130) . -26- i) Racial and Sexual Differences in Rates of Referrals Plaintiffs' exhibit 123 shows that, for the period from 1974 through 1978, whites have received a substantially higher rate of referrals than blacks, and that the disparity is generally statistically significant. In addition to comparing rates of referral for applicants generally, plaintiffs also compared rates of referrals for applicants in "available appli cant pools", i,e., applicants with the same occupational code as a job order or the same occupational code as someone actually referred on the job order. These are the same applicant pools used by the State defendants' expert. (Testimony of Michael Frodyma, Tr. 243-44). Thus, use of these "available applicant pools" compares applicants whom MSES has, in its assignment of occupational codes, con sidered comparably qualified. Plaintiffs' exhibit 123 shows the same pattern of statistically significant racial disparities in rates of referral for persons in "available applicant pools" as was shown for the general population. j) Referrals Out-of-Code When MSES referred an application to a job category with an occupa tional code outside the same initial-digit code "family" as the applicant's occupational code, the referral was to a job for which MSES had not considered the applicant best suited. Plaintiffs' exhibit 107 shows that 68 applicants were referred out-of-code to Clerical and Sales jobs in 1970, and that 85.3% of them were white. Mr. Lindsey testified that he did not know why, when blacks constituted more than 60% of the applicants in the office, 85.3% of the re ferrals out-of-code to Clerical and Sales jobs had been received by whites. (Tr. 1016-17). Plaintiffs' exhibit 108 shows that 68 applicants were referred out-of- code to Service jobs in 1970, and that 64.7% of them were black. Fifteen white women, and 18 black women, were referred to Service jobs despite having been -27- classified in Clerical and Sales codes. Plaintiffs' exhibit 109 focuses on the women referred out-of-code to Service jobs, and shows that only black women were referred out-of-code to the traditionally-black Domestic, Cook or Kitchen Helper jobs. Thirteen of the 15 white women--86.7%---with Clerical and Sales codes were referred to Waitress positions. Of the 18 black women with Clerical and Sales codes, however, 14--77.8% of the total-- were referred to Domestic or Cook positions. Plaintiffs' exhibit 75 shows that there were 113 parsons referred out-of- code to Domestic positions in 1971, and that 110 of them 97.3% of the total-- were black women. Two of them had been given Clerical and Sales codes. The district court did not address this evidence in its decision. k) Other Classwide Evidence of Discrimination in Referrals "Job development" is "the process of soliciting an employer's order for a specific applicant for whom the local office has no suitable opening currently on file." A local Employment Service office is supposed to engage in job development activities for applicants with unusual skills or training and for applicants who are hard to place, because few orders are received for the kinds of work they can do. (Federal defendants' exhibit 51, §§ 1685, 1687). Most of State defendants’ exhibits 44 through 52 contain racial breakdowns df job development activity; they show that the proportion of placements resulting from job development activities undertaken on behalf of black applicants is far lower than the proportion of applicants who are black. (Plaintiffs' exhibit 170) . Mr. Beverly testified that he would not have looked at these figures as indicating any cause for concern as late as 1977, but his office had become more sophisticated with the introduction of the "benchmark" analysis in the USES PEER program, and he would now look at it "a whole lot different". (Tr. 2294-95) . Mr. Lindsey testified on direct examination that job developments were -28- used for both skilled and unskilled jobs, and that such efforts should be under taken for applicants "who are hard to place because few orders are received for the kinds of work they can do", as well as for highly skilled applicants. (Tr. 2445). When confronted with the racial disparities shown on his own self appraisal forms and reflected in plaintiffs' exhibit 170, he changed his testi mony and explained them by saying that blacks had less experience and less training than whites, and job developments were undertaken only for persons with specific qualifications. He admitted that the decision whether to undertake job development efforts for a particular applicant was subjective, and that he had not always made specific inquiries of interviewers to find out whether they 21/ had been discriminating in job developments. (Tr. 2465-67, 2*478) . In theory, Federal regulations prohibit Employment Service offices from processing job orders from local employers known to be discriminatory. (Federal defendants' exhibit 51, § 1294; State defendants’ exhibit 8, § 1294; decision, R. 2140, App. 106). The district court found that these regulations were "legal and effective." (R. 2132, App. 98-99). Nevertheless, MSES's own investigation of the EEOC charges filed by the plaintiffs herein show that no black applicants referred to Baxter (Travenol Laboratories) would be hired unless they knew, and were recommended by, three particular black individuals: This is common knowledge in the community and anyone with whom you talk is knowledgeable about the procedures used to get employment at Baxter. 21/ Freddie Funchess, a black applicant, had taken accounting courses in high school and had finished two years of college in Business Administration. He had had jobs and training in a wide variety of occupational areas. (Tr. 486-500). Plaintiffs' exhibit 40, p. 1, shows that he visited the Bolivar County branch office on February 26, 1974, and was not given a referral then or later. Plaintiffs’ exhibit 4, vol. I, pp. 402-03, shows that a job development was successfully undertaken on behalf of a white applicant two days later, for a job as Manager Trainee for the Sonic Drive-In. The job required a high school degree, and did not require any previous experience. Mr. Lindsey did not know why a job development contact had not been made for Mr. Funchess. (Tr. 1033-37) -29- ... Applicants who exceed the norms and are other wise qualified for work are denied employment at Baxter simply because they are not known by the three men mentioned above. (Plaintiffs’ exhibit 116, p. 2; Tr. 2285-87). Despite this knowledge by MSES that Baxter imposed a condition on black applicants which it did not impose on white applicants, MSES continued to service job orders from Baxter. In its most recent investigation of record into the actions of the Bolivar County branch office, the Secretary's Office found on June 29, 1968, that the Cleveland branch office and eight other MSES offices had discriminated against blacks in making referrals to positions which had traditionally been closed to blacks: Each of these studies revealed patterns of white applicants being referred to public contact posi tions, in many cases with no related codes or experiences, over better qualified Negroes who were available for referral during the time that the orders were being serviced. (Plaintiffs’ exhibit llM- at p. 7) . The Department of Labor also found that the Bolivar County branch office of MSES had discriminated against blacks in refer ring applicants to summer employment positions at Baxter Laboratories. (Id. at p. 8). In his response, the defendant Aldridge did not dispute the general finding of discrimination in referrals but stated that the situation was improving. (Plaintiffs' exhibit 158 at p. 3). The district court did not address any of this evidence in its decision. 1).Dr. Malone’s Analysis Dr. Linda Malone, the State defendants’ statistician, testified that her analysis of referrals by the Bolivar County branch office for the period from 1974 through 1978 showed no discrimination. Her analysis was on a job-order- by-job-order basis; for each, she did separate analyses for each race, each sex, and each race/sex group. In each analysis, she examined each job order to -30- determine whether she would place it in Group 1, meaning that there is a low probability of doing better for the group in question, in Group 2, meaning that there is some chance of discrimination against the group in question but that it is unclear, or in Group 3, meaning that there is a high probability that more referrals should have been given to the group in question. In performing her analysis for each job order, she used "available applicant pools”, defined as described at 27, supra. For each of the groups she studied, and for each time period, she found that her analysis placed most jobs in Group 1 and only a very small number of jobs in Group 3. She accordingly concluded that there was no discrimination. (Testimony of Dr. Malone, Tr. 1390-99, 14-36-39; State defend- 22/ ants’ exhibits 63-68) . An unusual feature of Dr. Malone’s approach is that some job orders wind up in Group 1 for all of the groups being examined. (Tr. 14-96) . However, if all of the job orders falling within Groups 2 or 3-- i.e., those with a questionable probability or high probability that more of the group in question should have been referred-- it is apparent that even Dr. Malone's analyses show strong racial and sexual differences. For example, in the period of time from October 1977 through July 1978 for long-term jobs, there was a possible ques tion of discrimination against white males with respect to only 6.1% of job orders; for black males, there was such a question with respect to 10.4-% of 22/ Dr. Malone’s analysis resulted in comparing a group plaintiffs con tended to be discriminated against with another group containing persons who were also contended to be victims of discrimination. Thus, in assessing whether there was discrimination against black males, she did not compare their refer rals with those given to white males; she compared black male referrals with the combined referrals of a group consisting of black women, white women, and white males. She never compared the actions of MSES as between black females and either white males or white females; she compared the referrals given to black females with the combined referrals given to a group consisting of black males, white males, and white females. (Tr. 1393-94-, 14-88-90). Of necessity, such an approach would tend to conceal whatever racial or sexual disparities in referrals may exist. -31- job orders; for black females, 27.8%; for all blacks, 24.8%; and for all women, 23.0%. Table 12 on the next page provides the details. The essence of Dr. Malone’s analysis is that she looked at each job order separately. She admitted that her analysis would not detect discrimina tion if an "available applicant pool" was 50% black, and if MSES processed a hundred job orders for that pool, each time referring one white male for a total of 100 white males and no others referred. (Tr. 1524-25). If only one applicant were referred on a job order, that order would of necessity have been placed in Group 1 with respect to each race, sex, and race/sex group she studied. (Id.) In point of fact, MSES very often refers only one applicant on a job order, or closes a job order without any referrals: Table 13. No. of Job Orders Closed Without Referrals, or Closed With One Referral Year No. of Job Orders No. Closed Without Referral No. Closed With One Referral Percent Of Total 1970 273 18 90 39.6% 1971 494 36 191 46.0% 1972 797 27 340 46.0% (Calculation of counsel from plaintiffs’ exhibits 1, 2, and 3). Thus, Dr. Malone’s approach could not possibly find any problem in close to half of the 23/ job orders serviced by MSES. Dr. Mann, plaintiffs' expert, testified that Dr. Malone's approach was not appropriate. (Tr. 154-6-48, 1553-54). Finally, Dr. Malone's analysis assumed that there was no discrimination in classifying applicants into particular occupational codes. She admitted 23/ Dr. Mann testified, from a document prepared by Dr. Malone and earlier tendered to plaintiffs, that 30% to 40% of the job orders she used in her analysis had only one referral. The district court sustained an objection to the document’s admission, on the ground that going into the matter would take time (Tr. 1556-59), but no motion was made to strike this testimony. -32- -33- Table 12. Percent of Job Orders Falling Within Dr. Malone's Group 2 or Group 3 (Questionable Probability or High Probability That More Referrals to the Job Order in Question Should Have Been Given)___ ______________ Time Period Duration of Job Total No. of Job Orders White Males (SDX-65) Black Males (SDX-66) White Females (SDX-67) Black Females (SDX-68) All Blacks (SDX-64) All Women (SDX-63) 1/74-6/74 Short-Term 521 8.8% 12.9% 10.0% 15.5% 24.0% 11.5% 1/74-6/74 Long-Term 54 13.0% 22.2% 9.3% 18.5% 38.9% 16.7% 7/74-6/75 Short-Term 688 5.1% 12.8% 7.0% 16.4% 25.1% 11.0% 7/74-6/75 Long-Term 88 12.5% 22.7% 3.4% 23.9% 33.0% 25.0% 7/75-6/76 Short-Term 172 12.2% 23.8% 4.1% 15.7% 26.2% 16.9% 7/75-6/76 Long-Term 557 5.9% 17.1% 13.8% 14.2% 28.2% 9.9% 10/76-9/77 Short-Term 143 12.6% 15.4% 4.9% 4.9% 30.1% 23.8% 10/76-9/77 Long-Term 590 6.1% 14.7% 9.8% 9.8% 31.4% 11.7% 10/77-7/78 Short-Term 130 4.6% 6.9% 3.8% 35.4% 19.2% 33.8% 10/77-7/78 Long-Term 460 6.1% 10.4% 12.2% 27.8% 24.8% 23.0% that she had done nothing to check the accuracy of her assumption. (Tr. 14-83 84) . The district court did not mention her testimony or exhibits in its decision. m) The Defense that Job Openings Were Scarce The district court found that job openings on file in the Cleveland office were scarce, and that MSES employees did what they could to try to help applicants obtain whatever suitable work might be available. An examination of the job orders in evidence as plaintiffs' exhibits 1-3 show that a substan tial number are closed without referrals, are closed with only one referral, are closed with fewer referrals than there are openings, or are closed with the same number of referrals as there are openings. For the years 1970 through 1972, plaintiffs' exhibits 1-3 show: Table 14. Numbers of Referrals Made on Job Orders, 1970-1972 No. of No. of Job Orders in Which MSES Job Did Not Give the Employer a Percent of Year Orders Choice of Applicants to Hire Total 1970 273 118 43.2% 1971 494 268 54.3% 1972 797 415 52.1% (Calculation of counsel). See also Table 13, supra. The closing of a job order without a referral obviously harms both the employer who has been unable to fill its job through MSES and the unreferred applicants at MSES. The same is true when MSES refers fewer persons than there are openings (7 job orders in 1970, 25 in 1971, and 4 in 1972). When MSES refers only one person, or refers the same number of persons as there are openings, the employer is deprived of a choice of applicants which the employer is intended to have (Stipulation 5 47), and unreferred applicants are deprived of an opportunity to have the employer make its own judgment as to their ability to perform its jobs despite -34- any lack of formal education or experience, as well as being depri\ed of a 24/ broader sense of employment opportunities. The result of the figures shown in Tables 13 and 14- is shown in Table 1 at p. 8, supra: the decision by MSES whether or not to make a referral is often tantamount to the decision whether or not the applicant will be hired. 5. Evidence of Discrimination in the Classification of Applicants a) The Relationship Between Classification and Referral Part, but not all, of the explanation for plaintiffs' evidence of discri mination in referrals can be explained by the different occupational codes in which MSES has classified white applicants and black applicants, male applicants and female applicants. Although the district court found that the assignment of an applicant to one occupational code would not prevent that applicant's re- 25/ ferral to a job in another code, if a suitable job became available, (R. 2118, App. 85) , the evidence is uncontradicted that MSES policy is to give first pri ority of referral to those applicants meeting employers' requirements who have the same occupational code as that of the job on which referrals are being made, then to give priority to applicants with codes having the same initial digit as 24/ Obtaining a broader sense of employment opportunities may be more important for blacks, as a group, than for whites as a group, because the tradi tional exclusion of blacks from many types of jobs has left them with fewer avenues of informal information about various types of job opportunities. The fact that job opportunities in Bolivar County tend to occur in parts of the county other than that in which many blacks live underscores this conclusion. See text, supra at 6; plaintiffs' exhibit 114- at p. 3. 25/ The basis of this conclusion was incorrect. Adline Ward Brown was not referred to a job at Baxter Laboratories a week after the assignment of a "Domestic” code to her on June 22, 1970, but a year after that code had been assigned to her. Plaintiffs' exhibit 16 shows a July 1 referral to Baxter after an entry dated June 29, 1971, so that the July 1 referral was clearly in 1971, not in 1970. Plaintiffs' exhibit 2, volume III, p. 11(d), item 19, shows the referral of Ms. Brown to Baxter on July 1, 1971. Mr. Lindsey agreed that Ms. Brown was referred to Baxter a year after she had been classified in the Domestic code, not a week later. (Tr. 84-1-43). -35- that of the job in question, and only thereafter to give referrals to appli cants who have occupational codes with different initial digits than that of the job order. MSES officials agreed that the assignment of an occupational code to an applicant tends to restrict that applicants’ chances of referral to jobs with a similar code. (Federal defendants' exhibit 51, § 14-70; State de fendants’ exhibit 10, § 14-70; Stipulation, 5 58; testimony of Mr. Lindsey, Tr. 807-09, 872-73, 2171-72; testimony of George Nash, Tr. 2051, 2096). b) Th'e Process by Which Occupational Codes Are Assigned While a number of factors are supposed to be taken into account in assigning applicants to various occupational codes, (Federal defendants’ exhibit 51, §§ 1190-1225; State defendants’ exhibit 6, §§ 1190-1214-; decision, R. 2136- 39, App. 102-05), the actual practices of MSES are much more simple. The appli cant’s previous employment experience is the factor given primary considera tion in assigning an occupational code. Training is the next most important characteristic, and personal characteristics are assertedly given little weight. (Testimony of Mr. Lindsey, Tr. 821, 894, 2167-72). Mr. Beverly testified that excessive reliance may have been placed on prior experience in some cases, to the exclusion of other qualifications. (Tr. 2303-04;.State defendants’ ex- 26/ hibit 80) . 26/ Adline Brown, a black high school graduate who had taken a typing course in high school and had had previous experience as a babysitter--there was a dispute whether she had also had previous experience doing housework-- applied at MSES on June 22, 1970, and indicated that she preferred factory work. She was classified as a Domestic and was given no referrals for a year. (Testi mony of Ms. Brown, Tr. 521-26; plaintiffs’ exhibit 16; cf. testimony of Mr. Lindsey, Tr. 838-43). Leanner Terrell, a black applicant with a fourth-grade education, had experience as a Cotton Sampler at a cotton gin. The work is seasonal, and lasts for only two or three months a year. She applied to MSES in 1972, 1973, 1974 and 1975, and informed MSES that she wanted another type of work, such as factory work. She was classified as a Cotton Sampler on each of those applications and was not given any referrals. (Testimony of Ms. Terrell, Tr. 456-60, 466-73; plaintiffs' exhibit 28) . (footnote continued) -36- The district court found that Federal regulations require that personal characteristics, such as vocational interests and preferences, must be con sidered in assigning codes to applicants. (R. 2137, App. 103). Mr. Lindsey testified that such factors were given only limited weight in making a classifi cation decision. (Tr. 2168-69). On Ms. Terrell's application forms filled out on February 24-, 1972, and June 16, 1972, the space for the indication of the applicant’s interests is in the part of the form to be filled out by the inter viewer, and was left blank. (Plaintiffs’ exhibit 28). Mr. Lindsey admitted that he had never instructed his staff to ensure that they obtained this information from all applicants, and he could not say that his staff asked every applicant for this information if the applicant did not volunteer it. (Tr. 883-85) . Ms. Terrell denied having told MSES staff that she desired Domestic work when she applied on May 12, 1957, but the MSES interviewer fill ing out her form wrote that she preferred work in a private home. She was given several referrals to Domestic jobs, but did not accept them. (Tr. 4-63-66, 27/ M-74--75; plaintiffs' exhibit 28) . (Footnote continued from previous page) Mr. Lxndsey agreed that her applications had been filed after layoffs from her seasonal job, and that there was then no practical opportunity of referral to the Cotton Sampler job category in which she was coded. He did not know of any nonracial explanation for repeatedly assigning her a code based on experience in which there was no practical opportunity of referral, agreed that the assign- ment of that code would have restricted her chances of referral to another type of job, and agreed that there had never been any substantial number of whites classified in the Cotton Sampler code. (Tr. 864-76) . 27/ An applicant's expression of interest does not necessarily have any effect on the assignment of an occupational classification. Plaintiffs' exhibit 16 shows a June 22, 1970 application for Adline Ward Brown, which notes her preference for factory work but assigns her to the Domestic code; plaintiffs’ exhibit 4-0 contains a November 14-, 1978 application for Freddie Funchess, noting his preference for Security or Mechanic work, but assigning him to the Material Handler code; although Ollie White did not recall the application (Tr. 332-33, 34-0-4-1), plaintiffs’ exhibit 4-2 contains an applica tion filled out on December 16, 1970, noting Mr. White’s preference for Baxter Laboratories but classifying him as a Toll Collector II. Plaintiffs' exhibit 89 lists black women, classified as Maids (Domestics), whose appli cation forms showed a preference for other work. -37- The district court found that Federal regulations require MSES inter viewers to explain the operation of the local office, discuss possible codes, and inform applicants of the codes assigned, so that applicants will understand the method of selection. (R. 2137-39, App. 103-05; Federal defendants' ex hibit 51, §§ 1098-1100; testimony of Mr. Lindsey, Tr. 851). To the extent that this finding indicates that providing information to the applicant will help the applicant safeguard his or her interests, the applicant must be informed that the assignment to a particular occupational code will tend to restrict his or her chances of referral to other types of jobs. Without this information, the applicant would have no occasion to object to being assigned to a particular occupational code. The evidence is undisputed that applicants are not told that assignments to a particular occupational code may tend to restrict their 28/ opportunity of referral to other types of jobs, and might have objected if 29/ they had been so informed. The district court found that Mr. Lindsey "regularly gives talks in 28/ Testimony of Mr. Lindsey, Tr. 924-25. Mr. Nash testified that applicants agreed with their codes, (Tr. 2038, 2056), but went on to say that applicants were mainly concerned about getting a job, and 99% of them did not know what their code assignments meant. (Tr. 2057-58). While he said that he explained the meanings of codes to the applicants he interviewed (Tr. 2058, 2096)--only 10% of his time was spent interviewing after mid-1974 (Tr. 2039-41) --he could not speak for other interviewers, and was not surprised that several black applicants had testified they were not given such information (Tr. 2030- 81). Mr. Beverly testified that the applicants he had interviewed on his trips to the Cleveland office did not understand their occupational code assignments. (Tr. 2305-06; State defendants' exhibit 80, p. 2). 29/ Testimony of Rebecca Gillespie, Tr. 68-69, testimony of Mary Boyd, Tr. 138-39, 168; testimony of Ollie White, Tr. 337; testimony of Eva Ann Lofton, Tr. 350-52 (the Court sustained an objection to the question whether she knew that any code given her would reduce her chances of referral to another job); testimony of Pearline White, Tr. 391-92; testimony of Adline Ward Brown, Tr. 524; testimony of Freddie Funchess, Tr. 489, 495-96; testimony of Mary Alice Gillespie, Tr. 290-91, 308, 312-13 (she agreed only to be referred as a Kitchen Helper "if they couldn't find nothing else"); testimony of Leaner Terrell, Tr. 459. -38- the community ... about the Job Bank Viewers, how to use them, and the oppor tunities for employment that they contain." (R. 2125, App. 92) . The Job Bank viewers available to the public show information about available local jobs, and about available jobs in other parts of Mississippi. While the public is not given access to the names and addresses of employers, details of the available jobs and their pay rates are provided. The viewer uses micro fiche, and it is not possible to use it unless one has been shown how to do so. (Stipulation, 55 51-54; testimony of Mr. Lindsey, Tr. 797-99; 927-30, 933-40). Mr. Lindsey believed it was "useless" to show applicants how to use the Job Bank viewers if they are not available for referrals outside the area, but thought that most applicants were shown how to use the viewers. (Tr. 927-29). Mr. Nash agreed. (Tr. 2073-75). The use of a Job Bank viewer can help appli cants find jobs in codes to which they have not been assigned, and thus over come the restrictive effect of their code assignments, because it enables them to identify for themselves available local or distant jobs and the qualifica tions for such jobs, and to make their interest in such specific jobs known to MSES interviewers. (See Stipulation, 5 53; testimony of Freddie Funchess, Tr. 490, 492-93). However, a substantial number of black applicants, including one called as a witness for the State defendants, testified that the Bolivar 30/ County branch office did not show them how to use the Job Bank viewers. 30/ Testimony of Rebecca Gillespie, Tr. 74, 104; testimony of Eva Ann Lofton, Tr. 350; testimony of Pearline White, Tr. 392; testimony of Freddie_ Funchess, Tr. 490; testimony of Mary Alice Gillespie, Tr. 291, 310-11; testi mony of Leaner Terrell, Tr. 459; testimony of Ollie White, Tr. 337-38; testi mony of Gloria Triplett, Tr. 435; testimony of Elexo Warren (State defendants’ witness), Tr. 1990. Mr. Beverly agreed that some of the applicants with whom he discussed the operations of the Cleveland local office were not aware of the Job Bank viewer. He did not think interviewers were required to explain its use. Tr. 2310-11; State defendants’ exhibit 80, p. 2. -39- The district court found that Federal regulations require the assignment, "[w]here opportunities for employment are limited,", of "at least one additional classification for which there is a specific or general opportunity for work," (R. 2138, App. 104). The use of secondary occupational codes increases an applicant's opportunity for referral. (Stipulation, 5 41; testimony of Mr. Lindsey, Tr. 810). It was the policy of the U.S. Department of Labor to encourage the assignment of secondary occupational codes. (Stipulation of the parties, Tr. 813; Stipulation, 5 41). On June 29, 1968, however, the Sec retary's office found that the Cleveland branch office was not giving secondary codes to applicants, and found that this had a greater adverse effect on black applicants "because of their traditionally limited job opportunities." (Plain tiffs' exhibit 114, pp. 3-4). On August 19, 1968, Mr. Aldridge responded, stat ing that inadequate classification of applicants "was and continues to be state wide, one of our most serious problems." He said that additional training was being given. (Plaintiffs' exhibit 158, p. 3) . Plaintiffs' exhibit 106 shows that only 45 secondary codes were given to 3,292 applicants in 1970, a rate of only 1.4%. Questioned about this, Mr. Lindsey stated that it was the best his office could do, and showed "some pro gress". (Tr. 814-15). The same problem was noted in Mr. Beverly's EEO Evalua tion of the Cleveland office, dated November 7, 1978. (State defendants' ex hibit 80) . c) Classifications and Referrals in Service and Farinwork Occupations__________________________________________ One of the best examples of the interplay of discrimination in referrals with discrimination in classifications occurs with respect to the disproportion ate classifications of black women in, and referrals to, low-paid Service jobs (code 3). Plaintiffs' exhibits 60, 62, 64, 66, 68, 70, 72, and 74 show the -40- following information as to referrals. 31/ Table 15 . Referrals of Black Women to Service Occupations, 1970-1973 __________________ _______________ Referrals of Black Women To Service Occupations, 1970-1973 As Percent of All As Percent of Referrals Given to Year Women Referred Black Women_____ 1970 87.0% 1971 81.0% 1972 87.8% 1973 81.9% 65.9-% 92.8% 72.6% 98.8% Black women are also disproportionately classified into Service occupational codes, (plaintiffs' exhibits 80, 105), and the propriety of these referrals must be seen in light of the propriety of these classifications. Mr. Lindsey testified that he had received oral instructions in the mid- 1960's or late 1960's to avoid assigning "Maid" codes to applicants with some education. He stated that his office had tried to upgrade the classifications of blacks. (Tr. 829-26). Plaintiffs' exhibit 82 lists 90 black women and one white woman with an education of 8th grade or above, who were classified as Maids in 1970. Seven of them had high school degrees. Plaintiffs' exhibits 83, 89, and 85 showed that the same problems had continued in 1972, 1979, and 1976. Despite his instructions and the "upgrading" program, Mr. Lindsey testified that these assignments could have been proper, if the black women in question had had no other kinds of prior experience or if they had nad no other interests. (Tr. 829-30). When his attention was directed to the fact that these exhibits showed that a number of the black women reflected on them had had other kinds of prior experience, and that plaintiffs' exhibit 89 showed that a 31/ Ms. Thome, the preparer of these exhibits, testified that the column headings such as "white males referred" actually mean "white male referrals". (Tr. 697-52). -91- number of them had expressed preferences for other kinds of work, he testified that these classifications were mistakes which may have been made in haste. (Tr. 830-31). He testified that he had been aware of the problem, but that he had only cautioned his staff about the importance of upgrading codings, and thought it would be "getting pretty specific" actually to alter the codes; he surmised that MSES staff may simply have known that the codes were incorrect and not have relied on them. (Tr. 829-33) . However, half of the referrals made by the local office are made on the basis of a file search of the active applicant files, and these are maintained on different interviewers' desks according to the code numbers for which those interviewers are responsible, (Tr. 800-02, 939,40), and an interviewer handling another code may not even know of the existence of these black women mistakenly classified in a tradi- 32/ tionally-black occupation. Mr. Lindsey testified that he also made efforts to upgrade the classifi cation of black males assigned to the traditionally-black Farmwork codes. (Tr. 826). Plaintiffs' exhibits 86, 87, and 88 listed black males and white males classified in Service or Farmwork occupations, with an eighth-grade or better level of education. Mr. Lindsey testified that he saw the same problems indi cated by these exhibits as he had seen for the classifications of black women into Maid positions, but thought he saw progress. (Tr. 849). However, the up grading efforts were more successful for white males than for black males: 32/ One of Mr. Lindsey's self-appraisal forms state that Maid applica tions were pulled and that all persons classified into this occupational code had either farm experience, work experience, or no previous work experience. (There is no evidence that any inexperienced white women were routinely classi fied into this traditionally-black occupational code). Others state that the applications reviewed did not indicate any mistakes in coding. See State defendants' exhibits 44-55. The evaluative statements in these exhibits are not accurate. -42- Table 16. Results of Efforts to Upgrade the Codes of Male Applicants With an 8th-Grade or Better Level of Education, Classified in Service or Farmwork Codes, 1970-1976 ____ ________ __________________ Total No. of Such No. of Such White No. of Such Black Year Males Classified In Codes 3 or 4 Males Classified in Codes 3 or 4 Males Classified in Codes 3 or 4 % Black 1970 132 41 91 68.9% 1972 121 15 106 87.6% 1976 50 0 50 100 % (Calculation of counsel from plaintiffs’ exhibits 86, 87, and 88). d) Sharp Increases in the Numbers of Black Women Assigned to Clerical and Sales Codes After the Filing of Suit________ Plaintiffs’ exhibit 9 shows classifications in the traditionally-white Clerical and Sales codes in FY 1972, FY 1973, and FY 1974. While the exhibit does not indicate sharp changes in the numbers of black males assigned to such codes between FY 1972 and FY 1974, it does show such sharp changes for black women, far outstripping the changes of white women: Table 17. Occupational Code All Clerical and Sales Codes All Clerical Codes --Fully qualified --Not fully qual. -- Total All Sales Codes -- Fully Qualified --Not fully qual. -- Total Increases in the Numbers of Black Women Assigned to Clerical and Sales Codes, FY 1972 to FY 1974_______ White Females Number- FY 1972 Number- FY 1973 Number- FY 1974 % Change --% Change FY 1972-74 FY 1972-1974 254 321 507 + 100 % + 9.1% 151 117 147 - 2.6% - 13.7% 54 119 174 + 222.2% + 221.6% 205 236 321 + 56.6% + 12.7% 31 32 47 + 51.6% - 29.5% 18 53 139 + 672.2% + 110.7% 49 85 186 + 279.6% 0 % Accordingly, plaintiffs' exhibit 9 shows that black women were only 35.4% of the women assigned to Clerical and Sales codes in FY 1972, but were 50.1% of the women so assigned by FY 1974. While black women were only 39.1% of the Inexperienced women assigned to Sales codes (Codes 25X-29X) in FY 1972, they were 70.2% of such women in FY 1974. -43- There is no evidence of any event which could have caused a correspond ingly sharp increase in the qualifications of black women for Clerical or Sales jobs in this period, and in particular there is no evidence of such an event which could have had such an effect on the qualifications of black women while leaving the qualifications of white women relatively unchanged. The conclusion seems inevitable that MSES began evaluating the actual qualifica tions of black women much differently after the filing of suit. e) Racial Differences in the Classification of Female Applicants with a Seventh-Grade or Lower Level of Education, 197M-______ _________________________ _ Plaintiffs’ exhibits 77 and 78 show strong racial differences in assign ments to various occupational codes in 1971. While the above discussion shows that there were improvements for black women after the filing of suit, plain tiffs’ exhibit 81 shows that sharp differences continued to exist. This exhibit shows that these white women were 10.7% of the female applicants in 197h with a 7th-grade or lower level of education, but that these white women constituted M-2.9% of the women with this level of education who were assigned to Clerical and Sales codes, 31.3% of the women with this level of education who were assigned to Processing, Machine Trades, Bench Work, Structural Work, and Miscellaneous occupational codes (codes 5-9) , but were only 5.6% of the women with this level of education who were assigned to the lower-paid, traditionally-black Service and Farmwork codes. f) Other Evidence of Discrimination in Classifications In 1968, the U.S. Department of Labor found that the Cleveland office had classified inexperienced white applicants with codes indicating that they were fully experienced in public contact positions, but had given inexperienced -hh- The Department alsoblacks codes in "menial or traditional job areas." found that white applicants were given both a code for the desired goal occu pation and a code for the stopgap occupation, but black applicants were given only one or the other. The Department also noted that MSES staff members at the Cleveland office let a black applicant sit for an extended period of time without being served, whereas a white applicant who subsequently walked in the door was given immediate service. (Plaintiffs' exhibit lib, pp. 3, 4). Mr. Aldridge's response on the question of discriminatory classifications has been described at 40, supra. On the question of discrimiiatory handling of appli cants, he stated: "Follow-up visits indicate individuals are served in the 34/ order in which they enter the office." (Plaintiffs' exhibit 158, p. 2). Plaintiffs' exhibits 78 and 79 are lists of the 73 occupational classi fications in the Bolivar County branch office in which ten or more applicants had been assigned in 1971. (Plaintiffs' exhibit 77 has more detailed informa tion) . These exhibits show a strong pattern of racially and sexually segrega ted occupational assignments. Plaintiffs' exhibit 78 shows that one race accounted for 80% or more of the applicants classified in 34 of the 73 occupa tional codes. This is 46.6% of the total. For women, the segregation of 33/ 33/ Many of the inexperienced white women assigned to Clerical and Sales codes were given full codes, erroneously indicating that they were fully experienced. 34/ The testimony of some of the plaintiffs and class members for the earlier years covered by this case supports the findings of the Department of Labor. Mary Boyd testified that white applicants were given longer interviews than blacks. (Tr. 141-42, 161-62). Rebecca Gillespie testified that she was always treated courteously in the MSES office, but that interviewers spent much longer with white applicants than with black applicants. (Tr. 116-17, 120, 129 ~ 31). Pearline White testified that she would have to sit for 15 or 20 minutes in order to be waited on, but whites were handled as soon as they walked in. (Tr. 393). Leanner Terrell complained of the same practice, and also complained that MSES staff used courtesy titles when speaking to white applicants, but called black applicants by their first names. (Tr. 469-70). -45- occupational codes was even stronger. Plaintiffs' exhibit 79 shows that one sex accounted for 80% or more of the applicants classified in 56 of the 73 occupational classifications. This is 76.7% of the total. Moreover, one sex accounted for 90% or more of the applicants classified in 38 of the 73 occupa tional classifications. This is 52.1% of the total 73 occupational classifi cations . Plaintiffs' exhibit 15 is a list of black women assigned to nonclerical codes in 1970 who had clerical skills, clerical experience, or at least a tenth-grade education. Plaintiffs' exhibit 90 is a list of white women assign ed to Clerical and Sales codes in 1970. Mr. Lindsey's attention was drawn to the entries for a number of inexperienced white women on plaintiffs' exhibit 90 who had been classified in Code 2, including an inexperienced white woman with a second-grade education, (Tr. 898-906), ^ his attention was then drawn to the entries for a number of black women on plaintiffs' exhibit 15 who had better qualifications but were assigned instead to predominantly-black Service or Farmwork occupational codes. (Tr. 906-19). The Court sustained an objection to the question whether the qualifications of these black women were just as good as the qualifications of the white women given Clerical and Sales codes, on the ground that there might also have been some white women with the same 35/ types of qualifications who were not given Clerical and Sales codes. (Tr. 919) . The Court then sustained an objection to the question whether Mr. Lindsey knew of any reason, other than race, for the assignment of the previously-discussed 35/ Counsel for plaintiffs pointed out that the Service and Farmwork codes were almost entirely black in 1971 (plaintiffs' exhibit 77), and that he would like to ask Mr. Lindsey whether there was any appreciable difference in 1970. (If an occupational code is almost entirely black, there would be no appreciable number of white women in the same situation as the black women on plaintiffs' exhibit 15). The Court ruled that Mr. Lindsey should not be re quired to state his opinion with respect to what the statistics showed. (Tr. 920-22) . - 1+ 6- black women to predominantly-black occupational codes, while the previously- discussed white women had been assigned to Clerical and Sales occupational codes. (Tr. 922). The district court did not address this evidence in its decision, ex cept to state that "all of the relevant and available information was not reflected in the charts and schedules presented for use in the presentation of- 36/ plaintiffs’ case." (R. 2167, App. 130). The defendants presented no evi dence that the inclusion of any other factor would alter any inference to be drawn from the exhibits of record. 6. Discrimination in Testing General facts as to the testing practices of MSES, and as to policies and practices of the U.S. Department of Labor on the development and use of tests, are set forth in Stipulation 5? 76-109. Plaintiffs withdrew at trial any claim of discrimination with respect to the use of tests for counselling, and such use is not raised on this appeal. The Specific Aptitude Test Batteries ("SATB's") which are challenged on this appeal are the S-28 Table Worker te-st used for referrals to the Assembler job category at Baxter Laboratories from 1965 until November 3, 1971, the S-282 and S-282R Nurse Aide tests used for referrals to the Nurse Aide training program at East Bolivar County Hospital from November 3, 1971 to December 2, 1976, and the S-270 and S-270R tests used for referrals to the Licensed Practical Nurse training program at East Bolivar 36 / Exhibits containing all of the "available information" would have been impossible to prepare or, if prepared, to comprehend. Ms. Thome testi fied that, in some years, there were close to 9,000 pages of documents connected with applications, and that a total of 40,000 pages of MSES application and job order records had been microfilmed. Additional records were xeroxed. A total of eight persons worked on the duplication of MSES records in Mississippi over a three-week period. After the records were duplicated, a total of eleven temporary personnel were hired, in addition to permanent members of the Lawyers’ Committee’s staff, to analyze the records. The task began as soon as the copies from Mississippi arrived, and ended only a few hours before the October 1978 deadline for the exchange of exhibits. The preparation of the exhibits of record was extremely painstaking and time-consuming. (Testimony of Linda Thome, Tr. 539-40, 551-56, 762-63). -47- County Hospital from November 3, 1971 through at least the close of the record. (See Stipulation, 55 100-01). a) Evidence of Disproportionately Adverse Impact of the Challenged Tests on Blacks__________________________ The district court held that plaintiffs had failed to establish that any of the challenged tests had a racially disparate impact. (R. 2195-97, 2169, App. 110-12, 131) . Plaintiffs’ exhibit 127 shows that, from 1971 through 1976, 1,591 blacks were classified in the Nurse’s Aide code, but only 180 blacks were referred to the Nurse's Aide Training Program at East Bolivar County Hospital. This is a referral rate of 11.3% for blacks. The same exhibit shows that 278 others-- primarily whites-- had been classified in the Nurse's Aide code, but 137 of them 37/ had been referred to the training program, for a referral rate of 9-9.3%. The racial differences in the classification and referral rates are statisti cally significant. (Testimony of Dr. Outtz, Tr. 1583). The standard-deviation analysis for each of the years from 1971 through 1975 (there was only one referral in 1976), and for the total of these years, is shown in Table 18; Table 18 . Standard-Deviation Analysis of the Referrals to the Bolivar Nurse's Aide Training County Hospital Program at East Availability (Proportion of Those Classified as Nurse's 1971 1972 1973 1979 1975 1971-75 Aides Who Were Black) 73.5% 79.1% 87.0% 90.5% 87.9% 89.6% Sample Size (No. of Referrals) 63 60 39 85 69 316 Expected No. of Black Referrals 96.3 97.5 33.9 76.9 60.7 267.3 Observed No. of Black Referrals 30 35 20 99 96 180 Difference -16.3 -12.5 --13.9 -27.9 -19.7 -87.3 Standard Deviation No. of Standard Deviations Between Expected and Observed 3.5 3.1 2.1 2.7 2.7 6.9 Values -9.7 -9.0 -6.6 -10.3 -5.9 -13.6 (footnote 37 on next page) -98- The district court dismissed this evidence on the basis that "Dr. Outtz did not know how many if any of these persons were test selected." (R. 214-6, App. Ill). The record evidence contains no dispute on this score: it was not necessary to take and pass the Nurse’s Aide test in order to be classified as a Nurse’s Aide, but it was necessary to take and pass this test in order to be referred to the training program as shown on the exhibit. After making an initial mis statement, Dr. Outtz instantly corrected himself and so testified. (Tr. 1591). Mr. Lindsey testified that, when his office had an agreement with an employer that a test would be used, only persons who had passed the test would be re ferred, and persons with "H" scores would be referred before anyone with "M” 38/ scores would be referred. (Tr. 24-88-89, 24-91) . Plaintiffs’ exhibits 14-5 and 163 reflect information on a chart pre pared by MSES in 1971 and attached to an interrogatory answer, showing that blacks fell below the norms at a substantially higher rate than whites on the S-28 Table Worker test, and on a group of "all other SATB's" including the S-282 Nurse Aide test and the S-270R Licensed Practical Nurse test. Dr. Outtz testified that the racial disparity for the S-28 test was statistically signifi cant at the .02 level, and that the racial disparity for "all other SATB’s" was statistically significant at the .001 level. (Tr. 1582-83). The exhibit (Footnote from previous page) 37/ Ms. Thome testified that the "Training Program" mentioned in the ex hibit was the training program at the East Bolivar County Hospital. (Tr. 726). 38/^Stipulation exhibit 53 at p. 13 describes the scoring of SATB results. If an applicant’s scores meet or exceed the cutting scores for each aptitude test in the battery, the letter grade ”H" is assigned. If the applicant’s scores, when supplemented by one standard error of measurement ("SEM")--ranging from 6 to 12 points, depending on the aptitude--meet or exceed the cutting scores for each aptitude test in the battery, the letter grade "M", is assigned. If an applicant's scores do not meet the requirements for an "M”, the letter grade ’f’L" is assigned. Stipulation ? 82 states that this procedure is the one used. -49- itself did not show who was referred and who was not referred. (Id., Tr. 1590- 91). The district court rejected this evidence, stating that all applicants classified as "failing" in the exhibits were, according to testimony and De partment policy, eligible for referrals, that some of them scored "M", and that those scoring "M" received "very nearly the same consideration as those ’passing.T" (R. 2156, App. 120). The evidence, however, is undisputed that MSES never referred applicants with "L" scores. (Testimony of Mr. Lindsey, 39/ Tr. 29-91) . The refusal to refer applicants with "L" scores was not shown to be a violation of Department policy. Indeed, Mr. Hawk testified that "it would be a rather rare case that L’s would be referred". (Tr. 1866). Applicants with "M” scores were never referred unless there were too few applicants with "H" scores to meet the demand for referrals. (Testimony of Mr. Lindsey, Tr. 29-89-91). Thus, all of the applicants in the "Below Norm" statistics in plain tiffs’ exhibits 19-5 and 163 were, at best, disadvantaged by their test scores in obtaining referrals; at worst, they were barred entirely from referral. Plaintiffs’ exhibit 18 is a list of both black and white female appli cants assigned to the Licensed Practical Nurse code during 1972, 1979-, and 1976. While MSES kept some test results on application forms (e.g ., plain- 39/ The sole exception to this policy was that an applicant with an "L" score could be referred to an employer if the SATB in question used the G, V, or N aptitudes and the SATB had not yet been validated on members of minority groups. In such situations, the Department of Labor's "interim referral policy" was to apply, and the local office was to make referrals of enough mem bers of minority groups to ensure that there would be the same proportion of minorities among the referrals as the proportion of minorities in the local office’s applicant flow. If necessary to meet this requirement, minority ap plicants with "L" test scores could be referred. (Stipulation, 89-, 85; testimony of Mr. Lindsey, Tr. 2489-91). While the interim referral policy was supposed to have been applied to the Nurse’s Aide test until May 21, 1976 (plaintiffs' exhibit 165, p. 2), plaintiffs’ exhibits 126 and 127 show that it was not in fact applied. (Testimony of Mr. Lindsey, Tr. 29-96) . -50- tiffs’ exhibit 16; testimony of Dr. Outtz, Tr. 1592-99), much of its testing 90/ information is kept on test record cards. (Affidavit of Counsel, R. 2099- 2100; quotation by counsel for Federal defendants from a 1979 deposition of Mr. Lindsey, Tr. 1635-36). Test results for the Licensed Practical Nurse test were reported on the application forms of 35 whites and of 27 blacks re flected on this exhibit, and the exhibit contains their test scores. It shows the following: Table 19. Test Results for the Licensed Practical Nurse SATB Reported on Application Forms_____________________ Total ”H’’ Scores ’’M’’ Scores "L" Scores Whites: -- Number 35 30 5 0 -- Percent 100% 85.7% 19.3% 0% Blacks: -- Number 27 5 8 19 -- Percent 100% 18.5% 29.6% 51.9% Although this information was called to the attention of the district court in a special post-trial briefing of the issue and again in plaintiffs’ post-trial proposed findings of fact and conclusions of law, the district court did not discuss this evidence in its decision. The standard-deviation analysis of the information shown in Table 20 is as follows: 90/ Because test record cards do not record the race of the applicant whose test results are reported, it would have been an extremely difficult and time-consuming task to track down the applications of the persons reflected on these cards and obtain their race. Applications are kept with the records for the last year in which the applicant was active, so many years' data would have to be checked. In addition, women's marital name changes would complicate the task greatly. (Affidavit of counsel, R. 2099-2100). -51- Table 2Q . Standard-Deviation Analysis of the Test Results for the Licensed Practical Nurse SATB Reported on Application Forms "H" Scores "H” or "M” Scores "L" Scores Availability (Proportion of Blacks in the Group) 43.5% 43.5% 43.5% Sample Size (No. Achieving the Score Indicated) 35 48 14 Expected No. of Blacks 15.2 20.9 6.1 Observed No. of Blacks 5 8 14 Difference -10.2 -12.9 +7.9 Standard Deviation 2.9 3.4 1.9 No. of Standard Deviations Between Expected and Observed Values -3.5 -3.8 +4.2 The defendants stipulated to substantial differences in the average scores of blacks and of whites on each aptitude of the General Aptitude Test Battery, from which the challenged Specific Aptitude Test Batteries were drawn. (Stipulation, 5 78). The district court noted the argument of the Federal defendants that this nationwide data did not show any impact on blacks of any test in the Cleveland local office, (R. 2146, App. Ill), and seemingly accepted this argument. However, Mr. Lindsey testified at trial that he had noticed that test scores were "somewhat lower than normal" on the G, V, and N aptitudes. (Tr. 2502-03 (as corrected)). It is clear that Mr. Lindsey was referring to 41/ black scores. 41/ Counsel for plaintiffs asked the court reporter, Mr. Holman, to check his notes as to the accuracy of Tr. 2503, line 4, as originally reported, because this did not accord with counsel’s recollection or with Mr. Lindsey’s 1974 deposition testimony. Mr. Holman checked his notes, and stated that the phrase "lower than" had inadvertently been deleted from this line. He stated that he would provide a corrected page 2503 to the Court and to counsel. With this change, Mr. Lindsey's trial testimony is consistent with his March 19, 1974 deposition testimony that blacks scored "somewhat lower" than whites on the G, V, and N aptitudes. (Dep. Tr. 28). -52- The EEOC found that use of the GATB resulted in disparate impact against blacks. (Plaintiffs’ exhibit 103). Dr. Outtz testified that there was a consensus within the industrial psychological profession that blacks score much lower than whites on the kinds of aptitude tests exemplified by the GATB. (Tr. 1578-79). Dr. Outtz testified without contradiction that evidence such as the above would ordin arily be accepted within the industrial psychological profession as an ade quate showing of racially disparate impact. (Tr. 1583-84). b) Evidence as to Validation 1) Evidence Other Than Dr. Hunter's Testimony As a result of "outside pressure" and "legal events", the Department of Labor announced in 1972 a program to re-validate those SATB's using G, V, or N aptitudes, and to perform minority-differential studies on them. It concluded that other SATB’s would be unlikely to result in disparate impact against mem bers of minority groups. (Testimony of John Hawk, Tr. 1751-55, 1789-91; stipu lation exhibits 56, 57) . As a result of the re-validation program, the S-282 Nurse Aide test was replaced by the S-282R Nurse Aide test, using different aptitudes or cutting scores. For Licensed Practical Nurse, three different tests have been used between 1963 and the present. One was used from 1963 to 1970, when a new vali dation study led to the use of a different battery. That battery was again changed as a result of the re-validation program. Plaintiffs' exhibits 139 and 141 detail the differences in aptitudes and cutting scores each new study led the Department to make. Mr. Hawk testified that scores on the G, V, and N aptitudes tend to be highly intercorrelated, that the same is true for the perceptual aptitudes, that the same is true for the manipulative apitudes, and that there were strong -53- relationships across these groups. Because of this, it was unsurprising that use of different samples in different studies might produce different batteries. (Tr. 2580-81). However, plaintiffs’ exhibits 140 and 142 show that aptitudes found significantly related to performance in one administration of the test turned out to be unrelated to performance in another administration of the same test. Dr. Outtz testified that the lack of any stability in the results indi cates that the studies performed were not meaningful. (Tr. 2691-95). There are no minority-differential studies for the S-28 Table Worker test, the S-282 Nurse Aide test, or the S-270R Licensed Practical Nurse test. (Stipulation exhibits 70, 71 and 73; testimony of Mr. Hawk, Tr. 1800-01). Mr. Hawk stated that it was usual practice, in performing a new validation study, to re-run the old test on the new sample. He thought it "highly probable” that the old Nurse Aide and LPN tests had been run on the minority sub-samples used in the re-validation program, but admitted that the results were not set forth in the new technical reports and stated that he has no knowledge of the results of any such efforts. (Tr. 1867-68) . The Federal defendants did not produce any knowledgeable witness. There is a particular problem of record with respect to the Nurse Aide validation effort reflected in stipulation exhibit 68. The validation study was done for three hospitals in Virginia; while cross-validation samples were used, there were no minority sub-samples in the old data used for such purposes. Mr. Hawk testified to the Department’s policy of obtaining the largest possible validation samples, and of obtaining multistate validation samples. (Tr. 1682, 1684— 85, 1791-94). He also testified that it was improper to exclude data selectively from a sample in order to help obtain a correlation between test scores and the criterion of performance. (Tr. 1881). Plaintiffs’ exhibit 169 is a set of documents establishing that the Department had received data from -54- a large number of States which were not used in the re-validation program, and that the decision not to use these data cut more than 200 blacks and more than 200 whites from the validation sample. Mr. Hawk was unable to explain the ex clusion of this data from the published technical report, (Tr. 1885), and the Federal defendants’ right to provide an explanation later in the trial was re served. (Tr. 1886) . Although the Federal defendants recalled Mr. Hawk to the stand (Tr. 2578), he did not produce an explanation. Herbert Campbell, who was involved in the collection of data for the Nurse’s Aide study, stated that he also did not know why the data were excluded. (Tr. 2620-21). The Federal defendants did not produce any knowledgeable person or explanatory document. In order to determine whether test scores are related to performance, it is necessary to develop a criterion of performance. For the result of the study to be meaningful, it is essential that the criterion of performance be reliable, accurate, and meaningful. Prior to the Department's performance of minority-differential studies as part of its re-validation program, it most often used work samples, production records, and specifically detailed supervi sory ratings as criteria of performance. Where supervisory or instructors’ ratings were used, ratings were obtained from two different persons and corre lated so that their reliability could be checked. By contrast, the minority- differential studies in the re-validation program used supervisory or instruc tors' ratings almost exclusively, used a general "global” scale rather than one tied specifically to job duties, and used one rater rather than two. The same rater was asked to give ratings a couple of weeks apart, and those ratings were then correlated. This procedure overstates the actual reliability of these ratings. The use of the global scale makes it impossible to limit the ratings to the duties which are most important, and to exclude from the ratings those duties which are ordinarily learned in a short period of time by new hires. -55- The Department provides a range of 15 to 45 minutes training-- usually about 20 minutes-- to the raters, and relies on their previous experience as super visors in rating employees, but makes no effort to exclude from the data rat ings by supervisors without such experience. It has never researched the ade quacy of this amount of training. The validation sample is ordinarily obtained from a number of different locations, and there is no effort made to ensure that work which would produce one rating at one location is rated similarly elsewhere. The Department is aware that the same types of work are in fact rated differently in different locations. The dividing line between "poor" and "good" performance is set arbitrarily so that roughly a third of the work ers in any validation sample are considered "poor". This dividing point does not correspond to what the employers of workers in the validation sample would consider good or bad performance; because it is set nationwide, an employer with a small number of workers in the validation sample might have them all considered "good" or "poor" by the Department without regard to the employer's actual evaluation of them. Some of these workers had had lengthy service with 42/ their employers. Job duties also differ in some degree from location to location, and the Department's witnesses were unable to state with any partic ularity the degree of differences in duties which were tolerated in the vali dation samples. Although blacks generally receive lower supervisory ratings than whites, the Department relies on the twenty minutes' training to avoid racial bias in the ratings, and has never gone back to any supervisor who had rated blacks lower than whites and made any effort to find out whether the 42/ Although Mr. Hawk saw no problems with the idea that a long-term employee may be a bad employee for the purposes of a test validation study, (Tr. 1853-54), he testified that it was Department policy to forbid use of the resulting test on an applicant with prior experience in the job because of the applicant's "demonstrated competence". (Tr. 1732, 1876-77). -56- lower rating was objectively made. Where blacks tend to receive lower test scores than whites and tend to receive lower ratings than whites, any super visory bias in the ratings given to blacks may produce an erroneous finding of validity of the test under study. Generally, Mr. Hawk admitted, in the re validation program the Department gave a higher priority to the collection of data than to the development of a reliable criterion of performance. At the time of trial, the Department was researching more detailed rating scales, specifically tied to job duties, and using work samples and production records in some studies once more as criteria of performance. (Testimony of John Hawk, Tr. 1694, 1794-1800, 1801-16, 1822-25, 1831-40, 1842-58, 1864, 2578-79; testi mony of Herbert Campbell, Tr. 2597-99, 2608, 2621-24; testimony of Dr. Outtz, Tr. 2658-83, 2703-05, 2711-13; stipulation exhibit 80, p. 51; plaintiffs' ex hibit 166 at pp. 789-90, 793; plaintiffs' exhibit 167; plaintiffs' exhibit 172 at pp. 27, 34, 38, 43-44; plaintiffs' exhibit 173 at pp. 1397-1400). The validation studies in question suffer from the problem of restrictior in range. Although intended to be used on applicants, the studies were all performed upon experienced employees, some of whom had fairly long lengths of service. The workers in the validation samples generally had levels of educa tion considerably higher than those among the applicants in the Cleveland local office. In the validation sample for the S-270R75 Licensed Practical Nurse test, roughly a sixth of the black sub-sample and roughly a sixth of the white sub-sample were obtained from a hospital which hired only persons who were in the top 5% of their classes; the results of the validation sample were also un usual, in comparison to the results shown in other validation reports with minority subsamples, because blacks met the test norms in almost the same pro portion as whites. (Testimony of Mr. Hawk, Tr. 1860-67; testimony of Dr. Outtz. Tr. 2683-87, 2697-2700, 2706-09; plaintiffs’ exhibit 168, p. 19; plaintiffs' -57- exhibit 172, pp. 27, 38; plaintiffs; exhibit 173, pp. 1400, 1406-07; stipula tion exhibits 68-71). There is no concrete evidence of the practical utility of the challenged 43/ tests. The coefficients of correlation with the criteria of performance are not meaningful because the criteria themselves are not meaningful. In any event, the correlations are generally between .2 and .3, and this means that the scores on these tests predict no more than from 4% to 9% of the variability in performance among different employees, and leave 91% to 96% unmeasured. This is too small an effect to justify the use of a test with racially disparate impact. (Testimony of Dr. Outtz, Tr. 2687-91, 2702-03, 2720-21). Mr. Hawk thought that the test studies showed their utility, (Tr. 1869-71), but the test studies did not compare the effectiveness of the tests in selecting good workers, and avoiding the selection of bad workers or the exclusion of good workers, with the effectiveness of the other selection devices used by the employers in the study. Plaintiffs' exhibit 143 shows that the S-282R Nurse Aide test would have excluded 14.3% of the white "good students" and 27.5% of the black "good students". In assessing utility, this is an important con sideration. (Testimony of Mr. Hawk, Tr. 1869; testimony of Dr. Outtz, Tr. 2695-96). Dr. Hunter testified that this racial difference should be expected from a fair test. (Tr. 2906-08) . 2. Dr. Hunter's Testimony Dr. Hunter's testimony was crucial to the district court’s decision. It was based on mathematical operations he had performed on coefficients of correlation between test scores and criterion measures of performance in large numbers of studies done by others. He testified that his views were based on 43/ Dr. Hunter's testimony is discussed below at 63. -58- "1,700 correlations" done by himslef, his wife, and another (Tr. 2966), on "empirical evidence covering literally thousands of studies" (Tr. 3056), "hun dreds and hundreds of studies and a number of reviews on top" (Tr. 3097) , and, later, "1,100 sets of data" (not studies) from an unknown number of studies (Tr. 3111), all bearing on the question of minority differential validation. The number of studies on which he based his evidence on validity generaliza tion was equally astronomical. He had not himself read the studies on which he relied, but depended on the reports of these studies made by others when he performed his calculations. (Tr. 3111-12) . Because of his having been retainec by the Federal defendants-- at his July 5, 1979 deposition, he stated that he was first contacted by the Department of Labor about this case "last week", Dep. Tr. 5--plaintiffs have never had an opportunity to discover and test the accuracy of the data on which he relied, or to see whether it supports con clusions other than those to which he testified. In Dr. Hunter's testimony, a test was considered as having "validity" if there was a significant nonzero coefficient of correlation between test score and criterion of performance. A "validity coefficient" was no more nor less than a coefficient of correlation. (Tr. 3001, 3077). Thus, a test which had a .1 correlation between the test score and the criterion of performance-- explaining no more than 1% of the variability in individual performance and leaving 99% unmeasured-- could be considered to be valid and to have practical utility in his testimony. (Tr. 300H-05). Compare testimony of Dr. Outtz. (Tr. 2687-89) . Substituting the phrase "significant nonzero correlation" for the term "validity" for the sake of clarifying his testimony, Dr. Hunter testified that the large number of correlations he had reviewed showed a common pattern. Significant nonzero correlations between test scores and criteria of perfor -59- mance had been found for the same tests or even types of tests, even when job content changed radically. While the size of the correlation may differ sub stantially, some such correlations existed from job to job and from place to place within job. Dr. Hunter was unsure of the degree to which job content would have to change before there would be a zero correlation. He testified that the Department could safely end its test validation program for any job category in which there were two studies of adequate sample size, which found significant nonzero correlations for the sane job, where there was no signifi cant difference between the correlations found in the two studies. (Tr. 2973- 75, 2998-3001) . The district court accepted this testimony as establishing that TTthe validity of tests is broadly generalizable across jobs, geographical locations and applicant populations." (R. 2157, App. 122). Dr. Hunter testified that his research on "power" had led him to the conclusion that small sample sizes created as much as a 95% probability that an observed difference might not be found to be statistically significant when it was in fact a real difference, and not the product of chance. The larger the size of the sample, the greater the likelihood that the observed difference will turn out to be significant. (Tr. 2891-93; 29M-8-55) . The district court found that the tests challenged here were fair, and that the changes in aptitudes and cutting scores from one test validation effor' to another for the same jobs did not indicate invalidity of the tests. (R. 2158, 2160, App. 122-23, 12H). Dr. Hunter testified that a test should be considered fair if there is no statistically significant difference in the correlations of test scores and performance achieved by whites and those a- chieved by blacks, and testified that GATB test results should not be considerec unstable unless there are statistically significant differences between the correlations found in one study and those found in another. (Tr. 29M-5-55) . -60- Dr. Hunter did not apply his own research on "power" to these conclusions, to determine whether his finding of no statistically significant differences was simply a fluke stemming from small sample sizes. (Tr. 3069-7. Dr. Hunter testified that the 1,100 sets of data, or hundreds and hun dreds of studies, or literally thousands of studies, or 1,700 correlations-- as he variously described the basis for his opinion--he had examined for evi dence whether tests worked differently for members of minority groups than for whites had uniformly shown that there were no differences in correlations for blacks and for whites. (Tr. 2966, 3056, 3097, 3111). Shown a copy of plain tiffs' exhibit 193, which indicates that the S-282R Nurse Aide test excluded twice the proportion of blacks defined as "Good Students” than of whites de fined as "Good Students", he stated that this was not an indication of racially differential prediction in the test, but merely an indication of adverse impact which did not call the fairness of the test into question. (Tr. 2906-08, 3120-22). He explained that blacks have a greater degree of other variables making them good performers, which are not measured by the test. The test it self is an imperfect predictor of performance. (Id., Tr. 2909). He did not know whether these racial differences occurred because of genetic or environ mental factors. (Tr. 2899-95). Based on his research, Dr. Hunter testified that there was no longer any need for the provisions of the EEOC Guidelines requiring separate validation studies on each minority group for which the test is to be used. (Tr. 3097). However, in the course of defending another point, he inconsistently testified that many studies had been published finding that tests were valid for whites but not valid for blacks, and that such stud ies had been published even after issuance of the EEOC Guidelines. (Tr. 3117- 18) . The Department of Labor publishes technical reports when it believes it has successfully shown a test to be valid. Mr. Campbell testified that the -61- Department does not publish a report when the study fails to result in a find ing of validity. (Tr. 2614-15). Dr. Hunter’s attention was called to this testimony, and he was asked whether the accuracy of his findings was dependent upon the question whether the reviews of studies he examined were representa tive of all of the validation studies which had been done. He testified that it would make little difference to his findings if he had seen the results of a much greater proportion of successful studies than of unsuccessful studies, because "the bulk of unsuccessful studies” would have involved very small samples. (Tr. 3112-16). This was the conclusion he was supposed to have been checking. The district court found that there was no foundation for the allegation that the use of specific rating scales would reduce rater bias, and relied on Dr. Hunter for that point. (R. 2157, App. 121). Dr. Hunter stated that he thought specific rating scales were not superior to generalized rating scales, but stated that the research on which he relied had not examined the possible effect of specific rating scales on rater bias. He had not studied this issue. (Tr. 2934-) . While the district court found that racial bias by raters had not been demonstrated to exist in the validation reports at issue or as a general case, (R. 2156, App. 120), Dr. Hunter testified that the graph he drew at the bottom of Federal defendants’ exhibit 63, indicating the patterns of relation ships of test scores and criteria of performance in the data he had examined, was consistent with a pattern of blacks receiving lower supervisory ratings than whites because of subjective racial stereotypes by the raters. (Tr. 3122- 24) . The district court found that it was well established that restriction of range of the research sample "produces an underestimation of the relation ship found in the incumbent population from which the sanple was drawn." (R. 2158, App. 122). There is no disagreement in the record as to the truth -62- of this proposition in the abstract. However, it became clear in extended examination of the experts, in part by each other, that this proposition is true only if the research sample is in fact part of the population on which the test is to be used, and that there was no investigation whether the exper ienced employees in the test validation samples were in fact part of the same population for psychometric purposes as the inexperienced applicants on whom the tests are actually used. Dr. Hunter took the stand again to disclaim any reliance on his earlier statements about restriction of range. (Testimony of Dr. Hunter, Tr. 3079-87; testimony of Dr. Mann, Tr. 3139— 9-9; testimony of Dr. Hunter, Tr. 3152). The district court found that the tests in question here "evidence a high degree of utility." (R. 2160, App. 129-25). Dr. Hunter's calculations of utility assumed that employers could not do much better than random selec tion unless they used a test or educational requirement. (Tr. 2923-2M-) . When pressed on cross-examination, Dr. Hunter admitted that one could not reach a conclusion about the extent of the utility of a test unless one knew how mean ingful the distinction is between "good performance" and "poor performance" as measured by the test, and admitted that he did not have that data for the tests in issue. (Tr. 3010-11). 7. The Named Plaintiffs and the Class Member Witnesses The district court found that there was no discrimination against the named plaintiffs or against.the class member witnesses because they could not personally cite the referral of any white or male who was less qualified than they, or who was similarly qualified, to any job which they believed they could have performed. (R. 2117-18, App. 85-86). This finding may have been premised on the district court's further holding that plaintiffs had not proven classwide discrimination. -63- Treating the claims of plaintiffs and of class members as disparate treatment claims, they have established that they applied, that referrals were being made at the times of their applications and continued to be made there after, and that they were not referred to the more desirable types of jobs then available. The defendants did not introduce any evidence of a nondis- criminatory explanation in any of their cases. For example, plaintiff Rebecca Gillespie testified that she had an 8th-grade education, but obtained a 10th- grade G.E.D. equivalency certificate in 1971. She had taken the G.E.D. test in order to help her get a job at Baxter Laboratories, and re-applied at MSES as soon as she received her certificate. MSES told her that it had no refer rals to Baxter at that time. (Tr. 46, 4-8-49, 68-70). Mr. Lindsey testified that MSES was in fact making referrals to Baxter at the time of her applica tion. (Tr. 1003-04-) . Mr. Lindsey testified that, apart from the lack of a tenth-grade G.E.D. equivalency certificate, he did not know why any of the named plaintiffs could not have been referred to Baxter in 1969 when their Adult Basic Education program closed. (Tr. 951-52). Mary Boyd testified that she applied repeatedly at MSES, but was never offered a referral. She thought she could have done a job in which she had had no prior experience, and just wanted a chance to show that she could do it. Although MSES never informed her of the availability of Nurse’s Aide training, she learned about it through her daughter and a local official in 1971--years after she had started apply ing with MSES-- and with this knowledge was able to obtain a referral by MSES to the training program. (Tr. 135-36, 137a-40, 160, 166-67; stipulation ex hibit 20) . She depended on her lawyer to find out that whites received better jobs. (Tr. 155-56). The defendants never offered any evidence explaining theii failure to give her referrals prior to the referral to the Nurse’s Aide train ing program. Adline Ward Brown testified that a white friend of hers, Cindy -64- Blount, came into the MSES office at the same time she did in 1970, and was given a referral to Noel Industries while Ms. Brown was not given a referral. Ms. Brown had a twelfth-grade education, and Ms. Blount had only a tenth-grade education. (Tr. 521-22, 532-34-). Mr. Lindsey admitted that several job orders were available when Pearlie White made her applications, that they were in the types of work she was seeking, that white males were referred on them, and that he did not know why she was not referred. (Tr. 1037-4-4-) . He stated that job orders were available while Adline Brown’s applications were active, that 9 whites and 3 blacks had been referred on one, that 8 white men had been referred on another, and that he did not know why she was not referred. (Tr. 104-4— 4-7). These are illustrative examples. 8. Evidence on Class Determination Plaintiffs’ exhibit 150 is the transcript of a joint deposition of Mr. Hart and Mr. Ballard taken in 1977; it was designated as evidence at the February 27, 1978 hearing on class certification, and was ultimately received at trial. They testified that local office managers do not have discretion to deviate from MSES policies and procedures, follow the same policies in the assignment of occupational codes on the basis of prior experience, and evaluate experience in the same fashion. (Dep. Tr. 12-17, 54-55) . All local MSES offices follow the same policies with respect to the handling of employers’ educational and experience requirements, and do not differ in their restric tiveness when they try to negotiate educational requirements downward in the event that too few applicants with the desired level of education are available. (Dep. Tr. 31, 59-60) . All local offices have followed the same practices in determining whether an employer’s stated preference for employees of a particu lar sex was a BFOQ, and made their decisions in the same manner in which the Cleveland office made such decisions. (Dep. Tr. 73-74-). They all follow the -65- same policies on testing, and on the classification and referral of applicants. (Dep. Tr. 78-79) . Local offices commonly refer 4-,000 to 5,000 applicants a year to jobs in areas serviced by other local offices, and the availability of the Statewide Job Bank facilitates this movement. (Dep. Tr. 116-20). Dr. Ireland, plaintiffs' statistician, testified at the hearing that the computer printouts making up plaintiffs' exhibit 19 showed Statewide pat terns in common with the patterns of the Cleveland local office. (Hearing Tr. 42-78, 85, 88). Plaintiffs' trial exhibit 55, showing examples of educa tional requirements serviced by MSES offices across the State, was received at the hearing. (Hearing Tr. 123) . It shows the same types of educational re quirements as are listed in plaintiffs’ exhibits 5 and 56. And see plaintiffs' trial exhibits 131-135) . Summary of Argument Under the standards laid down by the Supreme Court, by this Court, and by other courts of appeals, plaintiffs have established a prima facie case of racial and sexual discrimination with respect to the manner in which MSES of ficials decide to classify and to refer applicants. The defendants have not produced statistically fair exhibits disproving the inferences to be drawn from plaintiffs' statistics, and have not shown any business necessity justi fying these results. Plaintiffs have also established that the State defend ants' uncritical acceptance and servicing of employers' educational and exper ience requirements, and their own use of experience and education in classify ing and referring applicants, has a racially and sexually disproportionate effect on members of the class. The defendants have not produced any evidence of the business necessity of these practices. The standard for establishing that a sexual preference is a bona fide occupational qualification is narrow, and has not been met here. -66- Mr. Lindsey and Mr. Aldridge had the responsibility of ensuring that the discriminatory practices shown at trial be avoided. Although placed on repeated notice of such problems, they failed to take any corrective steps effectively to stop such practices. They should be held personnaly liable. Plaintiffs have proven that the challenged tests had a racially dis parate impact against blacks, under the standards .laid down by this Court. The Uniform Guidelines on Employee Selection Procedures are an appropriate standard by which to judge the legality of a test which was still being used at the time they took effect. The tests have not been shown to be valid. In the alternative, the district court abused its discretion by allowing the Federal defendants to put a surprise expert witness on the stand, when the effect was to prejudice plaintiffs. Congress did not immunize the United States Employment Service from suit under Title VII when it passed the 1972 Amendments to that statute, and summary judgment was erroneously granted to the Federal defendants. The district court erred in failing to certify a Statewide class for the purposes of injunctive relief, and in failing to allow interventions to broaden the representation of such a class. ARGUMENT A. Plaintiffs Established a Prima Facie Case of Discrimination Which Has Not Been Rebutted The district court followed incorrect standards in deciding that no dis- 44/ crimination had been shown. The findings of nondiscrimination are not en titled to the protection of the "clearly erroneous" rule. 4-4-/ The lower court placed excessive reliance on provisions of the Em ployment Security Manual describing official policies, and too little reliance on evidence of what actually took place. It placed excessive weight on the fact that MSES has performed some welfare functions, and treated this as evi dence of nondiscrimination in classifications and referrals. It imposed on the named plaintiffs and on the class member witnesses the obligation of having (footnote continued) -67- Corley v. Jackson Police Dept., 566 F.2d 999, 1001-02 (5th Cir., 1978); Rowe v. General Motors Corp., 957 F.2d 598, 356 n.15 (5th Cir., 1972). More over, many of the district court’s findings are ultimate conclusions, rather than the types of subsidiary findings covered by Rule 52(a), F.R.Civ.P. Plaintiffs presented a substantial statistical case of racial and 95/ sexual discrimination. To assemble this quantum of proof by hand from the thousands of applications and hundreds of job orders filed each year was both extremely time-consuming and extremely expensive. The district court's re jection of this evidence because it did not contain "all of the relevant and available information", and its conclusion that statistics cannot present a true picture, is a rejection of the holding of Int'l Bhd. of Teamsters v. United States, 93.1 U.S. 329, 339-90 (1977) , that statistics are competent to prove discrimination in employment. By definition, statistics are a means of summarizing data so that inferences can be drawn from the data. It is not required that every possible showing be made, but merely that a credible show (Footnote continued from previous page) personal knowledge, independent of the evidence of record unearthed by their counsel, of named whites or males who were less qualified and were given bet ter treatment. In the one instance of discrimination it found, it held that MSES could escape liability because a third party had ceased using MSES just before the filing of a judicial Complaint. Reliance on the official policies is particularly misplaced in light of Mr. Beverly’s testimony that the policies had simply existed on paper in the early 1970’s, but that the "guard was being changed", the policies had actually started to be applied, and that he had "the feeling that this is the dawning of a new day." (Tr. 2319-20). 95/ The standard-deviation analyses presented in this brief were per formed according to the manner set forth in EEOC v. United Virginia Bank, 615 F.2d 197, 151-52 (9th Cir., 1980), and Hameed v. Int'l Ass’n of Bridge Workers, 637 F.2d 506, 513-19 (8th Cir., 1980). Vuyanich v. Republic Nat’l Bank of Dallas, 505 F.Supp. 229, 398-99, 350, and 358 n.158 (N.D. Tex., 1980) discussed the significance level of various standard deviations. A signifi cance level of .05--or one chance in 20-- is equivalent to 1.96 standard deviations. A significance level of .01 is the equivalent of 2.57 standard deviations. -68- ing sufficient to raise an inference be made. Phillips v. Joint Legislative Committee, 637 F.2d 1014, 1025 (5th Cir., 1981); Grant v. Bethlehem Steel Corp. 635 F.2d 1007, 1015 (2nd Cir., 1980) , cert, den., 49 U.S. Law Week 3926 (1981) . The State defendants cannot merely speculate that consideration of some un specified other factors might rebut the inference; they must present evidence 46/ so proving. The State defendants have not made the necessary showings. Dr. Malone's analysis is not probative of nondiscrimination because it tends to mask discrimination. Cf. James v. Stockham Valves & Fittings Co., 559 F .2d 310, 332 (5th Cir., 1977), cert, den., 434 U.S. 1034 (1978); Swint v. Pullman-Standard, 539 F.2d 77, 97) (5th Cir., 1976). The BFOQ exception to the prohibition of sex discrimination in Title VII is, as the MSES defendants were notified repeatedly beginning in 1967, "meant to be an extremely narrow" exception. Dothard v. Rawlinson, 433 U.S. 321, 334 (1977). The party seeking to rely on this exception must establish that "the essence of the business operation would be undermined" by failing to allow the preference. Diaz v. Pai American World Airways, 442 F.2d 385, 388 (5t Cir.), cert, den., 404 U.S. 950 (1971); Weeks y. Southern Bell Telephone & Telegraph Co., 408 F.2d 228, 234 (5th Cir., 1969) (the BFOQ defense requires more than simply "labelling a job 'strenuous'"). The State defendants' pecu liar view that Title VII did not prohibit sex discrimination for some years * * 46/ Falcon v. General Telephone Co, of the Southwest. 626 F.2d 369, * » l980) » vacated aad remanded on other issue, 49 U.S. Law Week 3743 (1981), opinion reinstated in relevant part, 647 F.2d 633 f5th~c7r~ !981); United States v. County of Fairfax, 629 F.2d 932, 940 (4th Cir "’l980) In Davis v. Califano, 6̂13 F.2d 957, 964 (D.C. Cir., 1979), the court of’ap peals held that a plaintiffs' prima facie case need not include statistical analyses of all of the factors relevant to promotion: The defending party, with the greater access to statistical evidence of other relevant factors, may utilize such evidence in its rebuttal presentation. -69- after 1965 is not a defense. Johnson v. Goodyear Tire & Rubber Co., 991 F.2d 1364, 1375-78 (5th Cir., 1974). Plaintiffs have identified a number of facially neutral practices of the State defendants which have been shown to have a disparate racial and/or 47/ sexual impact. These practices have tended strongly to reinforce the tra ditional racial and sexual patterns of local employment. Plaintiff Boyd testified: ... A lot of times people, they never know what a person can do if they never allow a chance. Because I went over to Greenville once to see about a job and they put me at the sewing factory, and they put me to a pressing machine, and I ran it all day, and I had never ran one before. (Tr. 160). These practices deny or restrict that chance. Moreover, plaintiffs' statistical evidence shows that the combined effect of these practices--the State defendants' actual application of their system--has a strong racially and sexually disparate impact against blacks and against women. Cf. Sledge v. J.P. Stevens & Co., 585 F.2d 625, 635-36 (4th Cir., 1978) , cert. den,, 440 U.S. 981 (1979), treating the defendant's personnel system as a "practice1' shown to have had disparate impact. The State defendants had the burden of proving that these practices were justified by business necessity. Griggs v. Duke Power Co., 401 U.S. 424 (1971); United States v. Georgia Power Co., 474 F.2d 906, 911-12 (5th Cir., 1973) (educational requirement); Parson v. Kaiser Aluminum & Chemical Corp., 575 47/ These are servicing of employers' educational aid experience re quirements without requiring the employer to produce facts showing the business necessity of the requirements, the State defendaits' heavy reliance on experi ence in classifying and referring applicants, their failure to inform black applicants about the meaning and restrictive effect of the codes to which they are assigned,their failure to show black applicants how to use the Job Back viewers, their failure to assign secondary codes, and their failures to make referrals. -70- F .2d 1374, 1390 (5th Cir., 1978), cert, den., 441 U.S. 968 (1979) (experience requirement); Baxter v. Savannah Sugar Refining Corp., 495 F.2d 437, 440-41 (5th Cir.), cert, den., 419 D.S. 1033 (1974) (the failure to advertise vacan cies may unlawfully limit the upward mobility of blacks). The district court's refusal to grant relief from the discrimination it found in the State defendants' servicing of Travenol Laboratories' educational requirement, on the ground that Travenol stopped using MSES shortly before the filing of the judicial Complaint, was erroneous. The Court clearly had juris diction over this practice, and was required to grant relief. Teamsters held that such a later change by a defendant: ... could be of little comfort to the victims of the earlier post-Act discrimination, and could not erase its obligation to afford relief to those who suffered because of it. 431 U.S. at 341-42. And see Parson, 575 F.2d at 1385-86; James, 559 F.2d at 325 n.18. Relieving the State defendants of all obligation to plaintiffs and their class is particularly inappropriate because of the continuation of the same practice with respect to other employers. The named plaintiffs and the class member witnesses have made out claims of discrimination in two separate ways. First, they met the four-prong test for establishing a prima facie case under McDonnell Douglas Corp. v. Green, 411 U.S. 792, 802 (1973). At that point, the defendants had the burden of producing admissible evidence of a legitimate, nondiscriminatory reason for their rejections, 411 U.S. at 802-03; Texas Dept, of Community Affairs v. Burdine, U.S. , 67 L.Ed. 2d 207, 216-17 (1981). Services provided at a later time are not a sufficient showing, Teamsters, supra; the evidence must speak to the time for which the claim of discrimination is made. The defend ants have not met their burden. -71- Second, the plaintiffs and class member witnesses are entitled to the benefit of a presumption that they were harmed by the classwide discrimination shown at trial. The State defendants had the burden of rebutting this pre sumption, and failed to meet it. Teamsters, 431 U.S. at 359-62; Sledge, 585 F.2d at 637. It was error to impose a higher burden on persons who testified. Sledge, 585 F.2d at 638. B. Plaintiffs Have Shown That the Challenged Tests Had Racially Disparate Impact and the Defendants Have Not Shown That They Were Valid The evidence of racially disparate impact of the challenged tests here in replicates each of the five types of proof held sufficient in James, 559 F .2d at 335-37, and exceeds them. If the evidence of disparate impact is con spicuous, plaintiffs "are not required to exhaust every possible source of evidence". Dothard v. Rawlinson, 433 U.S. at 33.1. Plaintiffs are not required to prove disparate impact "with complete mathematical certainty." Vulcan Society of N.Y.C. Fire Dept, v. Civil Service Comm'n, 490 F.2d 387, 393 (2nd Cir., 1973). Continuing, Judge Friendly quoted Justice Holmes: "Certainty generally is illusion, and repose is not the destiny of man." (Footnote omitted). Cf. Vuyanich, 505 F. Supp. at 355-56, noting that imposing onerous standards for proof of disparate impact would "effectively eviscerate the ’business purpose' requirement first set forth in Griggs". In evaluating the evidence on test validation, the slipshod nature of the Department's work counts heavily against it. Vulcan Society, 490 F.2d at 395-96. The fact that the Department relaxed its standards during the period of its minority re-validation program is particularly troubling. So, too, are the Department's unexplained decision to jettison a great deal of the data available to it for the Nurse's Aide re-validation effort, and its un explained decision to present no evidence concerning the result of running the old Nurse's Aide and LPN tests on data for the minority sub-sample it obtained -72- in the re-validation effort. The importance of the questions at stake, and the fact that Mr. Hawk had had time to discover the answers before returning to the stand, makes it difficult to avoid the conclusion that production of 48/ this information would have been harmful to the Department's case. In evaluating the evidence on validation, this Court should consider not only the standards of the EEOC Guidelines, but also those of the Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. §§ 1607.1 ejt seq. The trial court upheld the Department’s objection to use of the 1978 Uniform Guide lines at trial, on the basis that the validation studies in question had pre dated the Guidelines. (Tr. 1598-99, 1646-47, 2641-45). Indeed, the Federal defendants even objected, unsuccessfully, to consideration of the EEOC Guide lines at one point. (Tr. 2652) . This Court applied the 1978 Uniform Guidelines to studies performed during the period 1970-75 in Ensley Branch, of NAACP v. Seibels, 616 F.2d 812, 816-17, nn. 11, 12 (5th Cir.), cert, den., 66 L.Ed.2d 603 (1980) . Dr. Hunter, whose theories are more novel than the Uniform Guidelines, should have been barred from testifying, and his testimony should not be con sidered by this Court. In Shelak v. White Motor Co., 581 F.2d 1155, 1159-60 (5th Cir., 1978), this Court held that the trial court abused its discretion by permitting a "trial by ambush" which deprived the opposing party of an 48/ Geller v. Markham, 635 F.2d 1027, 1033 (2nd Cir., 1980) ("A trial court is best able ... to draw inferences from different parties' failure to include relevant statistics that are available to them ... .") ; Interstate Circuit v. United -States, 305 U.S. 208, 226 (1939) ("The production of weak evidence when strong is available can lead only to the conclusion that the strong would have been adverse. ... Silence then becomes evidence of the most convincing character."); Ward v. Apprice. 6 Mod. 265 (Q.B., 1705) (" ... [i] f very slender evidence be given against him, then, if he will not produce his books, it brings a great slur upon his cause.").; 2 Wigmore, A Treatise on the Anglo-American System of Evidence §291 at 187 (3rd ed. , 1940)";' 2 Conrad, Modern Trial Evidence §960 at 169 (1956). -73- opportunity to prepare adequately for the trial. While plaintiffs believe that the present reocrd requires reversal on the issues of test validation, this Court may not agree. If so, this demonstrates the prejudice to plaintiffs of allowing Dr. Hunter to testify; it is hard to engage in the most effective possible cross-examination if the witness can retreat into claims of thousands of studies whenever pressed, and counsel has no idea what difficulties there j+9/ may be for the witness in the data on which he relied. The lack of any evidence of a study of test fairness for the S-28 Table Worker test, the S-282 Nurse Aide test in use from 1971 to 1976, and the LPN test in use from 1971 to 1976, is a serious defect. EEOC Guidelines, 29 C.F.R. §§1607.9-(a) , 1607.5(b)(5) (1972); Uniform Guidelines, 29 C.F.R. §1607.19 (b) (8) (1980) ; American Psychological Association, Standards for Educational & Psycho logical Tests (1974), plaintiffs’ exhibit 172, §E9 at pp. 43-44. The Supreme Court emphasized the importance of such a study in Albemarle Paper Co. v. Moody, 422 U.S. 405, 435 (1975), and this Court considered the absence of such a study to be an important factor in rejecting validation studies in United States v. Georgia Power Co., 474 F.2d 906, 913-14, 916 (5th Cir., 1973), and United States v. Jacksonville Terminal Co., 451 F.2d 418, 455-56 (5th Cir., 1971) , cert. den., 406 U.S. 906 (1972) . Accord, United States v. City of Chicago, 549 F.2d 415, 430 (7th Cir.), cert, den. , 434 U.S. 875 (1977) (find ing a correlation for whites but none for blacks); Rogers v. Int’l Paper Co., 510 F.2d 1340, 1350 (8th Cir.), vacated on other grounds, 423 U.S. 809 (1975), modified in other respects, 526 F.2d 722 (8th Cir., 1975); Boston Chapter NAACP v. Beecher, 504 F.2d 1017, 1026 (1st Cir., 1974), cert. den., 421 U.S. 910 49/ See also Burns v. Thiokol Chemical Corp., 483 F.2d 300, 306-07 (5th CirT, 1973), noting the extent to which discovery limitations may have hampered effective cross-examination. -74- (1975). Dr. Hunter’s point that "validities” are the same for all groups must be rejected. Rogers, supra. Cf. Ensley Branch, in which this Court noted that test scores were significantly positively correlated with experimental ratings for firefighters with less than three years’ experience, but significantly negatively correlated with ratings for more experienced firefighters. For the sake of brevity, and because the facts themselves speak so loudly, plaintiffs will address their remaining points on validation summarily. First, the use of supervisory rating forms open to divergent interpretation was criticized in Albemarle Paper, 422 U.S. at 9-32-33. See EEOC Guidelines, 29 C.F.R. §1607.5 (b) (4) (1972); Uniform Guidelines, 29 C.F.R. §1607.14 (b) (2) ; United States v. City of Chicago, 549 F.2d at 431. Second, the long line of cases in this Court finding that the exercise of subjective judgments by whites tends to result in blacks being disadvantaged, e,g,, James, 559 F.2d at 328-29; Rowe, 457 F .2d at 359, demonstrate that care must be taken to guard against rater bias. Albemarle Paper criticized the lack of such care in the study before it, 422 U.S. at 432 n. 30. Third, Albemarle criticized validation of the test on experienced employees when it would be applied to inexperienced persons, 422 U.S. at 435. EEOC Guidelines, 29 C.F.R. §1607.5 (b) (1) (1972); Uniform Guidelines, §14 (B) (4) . C. The District Court Erred in Granting Summary Judgment for the Federal Defendants ■ . The district court ruled that the Equal Employment Opportunity Act of 1972, Pub.L. 92-261, 86 Stat. 103, withdrew the United States Employment Ser vice from the coverage of Title VII. (App. 64-66). The relevant provisions, before and after the amendments, are set out in the Brief Appendix below. As originally enacted, §701 (c) defined the term "employment agency" as -75- including any "person" regularly undertaking to procure employees for an employer, "but shall not include an agency of the United States ... except that such term shall include the United States Employment Service and the system of State and local employment services receiving Federal assistance." Thus, the only mention of USES was in an exception to an exclusion from the statute’s coverage. In 1972, Congress struck out the entire exclusion for governmental agencies, and defined "person" in § 701 to include governmental agencies as well as individuals and private organizations. The net effect of the 1972 amendments to § 701 was to extend Title VII coverage to governmental activities at all levels; there is no legislative history indicating any intent to re strict any level of the statute's coverage. While the district court found it significant that the United States Employment Service was no longer mentioned by name, the same is true for State Employment Services such as MSES. Both names were dropped in § 701(c) because they had originally been in merely as exceptions to an exclusion. The legislative history supports this interpretation. While the 1972 amendments were pending, the Senate Subcommittee on Labor of the Committee on Labor and Public Welfare issued a Report, "Proposed Equal Employment Oppor tunities Enforcement Act of 1971, S. 2515, S. 2617, and H.R. 1746, Bill Texts, Section-by-Section Analyses, Changes in Existing Law, Comparison of Bills In troduced" (1971). The Report is reprinted in the same Subcommittee’s Legis lative History of the Equal Employment Opportunity Act of 1972 (1972) at 1935. The Report stated at 105, Legislative History at 2041: The bill would eliminate the existing law's exemption of all levels of government from the definition of "em ployment agency". Consequently, to the extent that "an agency of a State or political subdivision of a State" (in addition to "the United States Employment Service and the -76- System of State and local employment services receiving Federal assistance" presently covered by the law) "regu larly" procures employees for a covered employer or which "regularly" procures for employees opportunities to work for covered employers, they would be covered.) (See 2(3) .) The district court's suggestion that § 717 of the Act provides an ex clusive remedy is misplaced, because the claims of applicants for employment referrals who are harmed by practices of USES are not covered by that section. By its terms, it applies only to Federal employees and applicants for Federal employment. D . The District Court Erred in Failing to Certify a Statewide Class The sole purpose for which plaintiffs sought a Statewide class was to obtain broad injunctive relief ending, in each MSES office, any practice shown at trial to be discriminatory and which was a result of Statewide policies or practices. The district court evidently found this idea manageable, because it stated at the class certification hearing that it would enter such Statewide relief if it found such discrimination arising as a result of Statewide poli cies, even though it was denying a Statewide class. After this Court’s de cision in Payne v. Travenol Laboratories, 565 F.2d 895, 898, (5th Cir.), cert. den., 939 U.S. 835 (1978), it is unclear whether a district court may enter broad injunctive relief when no named plaintiff or class member will directly benefit. The certification of a Statewide class is the only means of ensuring the availability of relief. The desirability of Statewide relief should not require extended dis cussion. If a test is found unlawful, its use should not be continued in other MSES offices; if the policy of accepting and servicing unvalidated educational and experience requirements set by employers is found unlawful, procedures will have to be developed to ensure an end to the practice, and they should be applied throughout the State, not just in Bolivar County. -77- The applicants for intervention should have been allowed to intervene as additional class representatives to support certification of the limited Statewide class sought. CONCLUSION For the reasons stated above, plaintiffs respectfully pray that this Court reverse the judgment of the district court, find that plaintiffs have shown classwide discrimination in the respects shown, and remand the case for the entry of relief. Respectfully submitted, WILLIAM L. ROBINSON RICHARD T. SEYMOUR LESTER GOLDNER Lawyers’ Committee for Civil Rights Under Law 520 Woodward Building 733 Fifteenth Street, N.W. Washington, D.C. 20005 NAUSEAD STEWART Lawyers' Committee for Civil Rights Under Law 720 Milner Building 210 South Lamar Street Jackson, Mississippi 39201 DOROTHY A. WINSTON 829- Second Avenue, North Columbus, Mississippi 39701 Dated: July 21, 1981 -78- A P P E N D I X Sec. 701 of the Civil Rights Act of 1964, 78 Stat. 253-54, provides in pertinent part: Sec. 701. For the purposes of this title--- (a) The term "person" includes one or more individuals, labor unions, partnerships, associations, corporations, legal representatives, mutual companies, joint-stock companies, trusts, unincorporated organizations, trustees, trustees in bankruptcy, or receivers. * * * (c) The term "employment agency" means any person regularly undertaking with or without compen sation to procure employees for an employer or to pro cure for employees opportunities to work for an employer and includes an agent of such a person; but shall not include an agency of the United States, or an agency of a State or political subdivision of a State, except that such term shall include the United States Employment Service and the system of State and local employment services receiving Federal assistance. Sec. 2 of the Equal Employment Opportunity Act of 1972, Pub.L. 92-261, 86 Stat. 103, provides in pertinent part: Sec. 2. Section 701 of the Civil Rights Act of 1964 (78 Stat. 253; 42 U.S.C. 2000e) is amended as follows: Cl) In subsection (a) insert "governments, governmental agencies, political subdivisions," after the word "individuals". * * * (3) In subsection (c) beginning with the semi colon strike out through the word "assistance". Sec. 703Cbl of the Civil Rights Act of 1964 provides: Cbl It shall be an unlawful employment practice for an employment agency to fail or refuse to refer for employment, or otherwise to discriminate against, any individual because of his race, color, religion, sec, or national origin, or to classify or refer for la - employment any individual on the basis of his race, color, religion, sex, or national origin. The EEOC Guidelines on Employee Selection Procedures, 29 C.F.R. §§ 1607.1 et seq. (1972) provide; PART 1607— GUIDELINES ON EM PLOYEE SELECTION PROCEDURES Sec. 1607.1 Statement of purpose. 1607.2 “Test" defined. 1607.3 Discrimination defined. 1607.4 Evidence of validity. 1607.5 Minimum standards for validation. 1607.6 Presentation of validity evidence. 1607.7 Use of other validity studies. 1607.8 Assumption of validity. 1607.9 Continued use of tests. 1607.10 Employment agencies and employ ment services. 1607.11 Disparate treatment. 1607.12 Retesting. 1507.13 Other selection techniques. 1607.14 Affirmative action. AUTHoamr: The provisions of this Part 1607 issued under sec. 713, 78 Stat. 265; 42 U.S.C. 2000e-12. Source: The provisions of this Part 1607 appear at 35 F_R. 12333, Aug. 1, 1970, unless otherwise noted. § 1607.1 Statement of purpose. (a) The guidelines in this part are based on the belief that properly vali dated and standardized employee selec tion procedures can significantly con tribute to the implementation of non- discriminatory personnel policies, as required by title VH. It is also recognized that professionally developed tests, when used in conjunction with other tools of personnel assessment and complemented by sound programs of job design, may significantly aid in the development and maintenance of an efficient work force and, indeed, aid in the utilization and conservation of human resources generally. (b) An examination of charges of dis crimination filed with the Commission and an evaluation of the results of the Commission’s compliance activities has revealed a decided increase in total test usage and a marked increase in doubtful testing practices which, based on our experience, tend to have discriminatory effects. In many cases, persons have come to rely almost exclusively on tests- as the basis for making the decision to hire, transfer, promote, grant member-- ship, train, refer or retain, with the result that candidates are selected or re jected on the basis of a single test score. Where tests are so used, minority can didates frequently experience dispropor tionately high rates of rejection by fall ing to attain score levels that have been established as minimum standards for qualification. It has also become clear that in many instances persons are using tests as the basis for employment decisions without evidence that they are valid predictors of employee job performance. Where evidence in support of presumed, rela tionships between test performance and job behavior is lacking, the possibility of discrimination in the application of test results must be recognized. A test lacking demonstrated validity (i.e., having no known significant relationship to job behavior) and yielding lower scores for classes protected by title VH may result in the rejection of many who have neces sary qualifications for successful work performance. (c) The guidelines in this part are designed to serve as a workable set of standards for employers, unions and employment agencies in determining whether their selection procedures con form with the obligations contained in title VH of the Civil Rights Act of 1964. Section 703 of title VXI places an affirma tive obligation upon employers, labor unions, and employment agencies, as defined in section 701 of the Act, not to discriminate because of race, color, religion, sex, or national origin Subsec tion (h) of section 703 allows such per sons “* * * to give and to act upon the results of any professionally developed ability test provided that such test, its administration or action upon the results is not designed, intended or used to dis criminate because of race, color, religion, sex or national origin.” § 1607.2 “Test” defined. For the purpose of the guidelines in this part, the term “test” is defined as any paper-and-pencil or performance measure used as a basis for any employ ment decision. The guidelines in this part apply, for example, to ability tests which are designed to measure eligibility for hire, transfer, promotion, membership, training, referral or retention. This defi nition includes, but is not restricted to, measures of general intelligence, mental _ ability and learning ability; specflc intel lectual abilities: mechanical, clerical and other aptitudes; dexterity and coordina tion; knowledge and proficiency; occu pational and other interests; and atti tudes, personality on temperament. The term “test" includes ail formal, scored, quantified or standardized techniques of assessing job suitability including, in addition to the above, specific qualifying or disqualifying personal history or back ground requirements, specific educa- 2a tional or work history requirements, scored interviews,- biographical informa tio n blanks, interviewers’ rating scales, scored application forms, etc. § 1607.3 Discrimination defined. The use of any test which adversely affects hiring, promotion, transfer or any other employment or membership opportunity of classes protected by title VH constitutes discrimination unless:, (a) the test has been validated, and evi dences a high degree of utility as here inafter described, and (b) the person giving or acting upon the residts of the particular test can demonstrate that al ternative suitable hiring, transfer or promotion procedures are unavailable for his use. § 1607.4 Evidence of validity. (a) Each person using tests to select from among candidates for a position or for membership shall have available for inspection evidence that the tests are being used in a manner which does not violate § 1607.3. Such evidence shall be examined for indications of possible discrimination, such as instances of higher rejection rates for minority can didates than nonminority candidates. Furthermore, where technically fea sible, a test should be validated for each minority group with which it is used; that is, any differential rejection rates that may exist, based on a test, must be relevant to performance on the jobs in question. (b) The term ‘'technically feasible" as used in these guidelines means having or obtaining a sufficient number of mi nority individuals to achieve findings of statistical and practical significance, the opportunity to obtain unbiased job per formance criteria, etc. It is the responsi bility of the person claiming absence of technical feasibility to positively demon strate evidence of this absence. (c) Evidence of a test's validity should consist of empirical data demonstrating that the test is predictive of or signifi cantly correlated with important ele ments of work behavior which comprise or are relevant to the job or jobs for which candidates are being evaluated. (1) If job progression structures and seniority provisions are so established that new employees will probably, within a reasonable period of time and in a great majority of cases, progress to a higher level, it may be considered that candidates are being evaluated for jobs at that higher level. However, where job progression is not so nearly automatic, or the time span is such that higher level jobs or employees’ potential may be expected to change in significant ways, it shall be considered that candi dates are being evaluated for a job at or near the entry level. This point is made to underscore the principle that attainment of or performance at a higher level job is a relevant criterion in validating employment tests only when there is a high probability that persons employed will in fact attain that higher level job within a reasonable period of time. (2) Where a test is to be used in dif ferent units of a multiunit organization and no significant differences exist be tween units, jobs, and applicant popula tions, evidence obtained in one unit may suffice for the others. Similarly, where the validation process requires the col lection of data throughout a multiunit organization, evidence of validity specific to each unit may not be required. There may also be instances where evidence of validity is appropriately obtained from more than one company in the same in dustry. Both in this instance and in the use of data collected throughout a multi- unit organization, evidence of validity specific to each unit may not be re quired: Provided, That no significant differences exist between units, jobs, and applicant populations. § 1607.5 Minimum standards for vali dation. (a) For the purpose of satisfying the requirements of this part, empirical evi dence in support of a test's validity must be based on studies employing generally accepted procedures for determining cri terion-related validity, such as those described in “Standards for Educational and Psychological Tests and Manuals" published by American Psychological Association, 1200 17th Street NW„ Washington, D.C. 20036. Evidence of content or construct validity, as defined in that publication, may also be appro priate where criterion-related validity is not feasible. However, evidence for con tent or construct validity should be ac companied by sufficient information from job analyses to demonstrate the rele vance of the content (in the case of job knowledge or proficiency tests) or the construct (in the case of. trait measures). Evidence of content validity alone may be acceptable for well-developed tests that consist of suitable samples of the essential knowledge, skills or behaviors composing the job in question. The types of knowledge, skills or behaviors con templated here do not include those which can be acquired in a brief orien tation to the job. (b) Although any appropriate valida tion strategy may be used to develop such empirical evidence, the following minimum standards, as applicable, must be met in the research approach and in the presentation of results which con stitute evidence of validity: (1) Where a validity study is conducted in which tests are administered to appli cants, with criterion data collected later, the sample of subjects must be represent ative of the normal or typical candidate group for the. job or jobs in question. 3a This further assumes that the applicant sample is representative of the minority population available for the job or jobs in question in the local labor market. Where a validity study is conducted in which tests are administered to present em ployees. the sample must be represent ative of the minority groups currently included in the applicant population. If it is not technically feasible to include minority employees in validation studies conducted on the present work force, the conduct of a validation study without minority candidates does not relieve any person of his subsequent obligation for validation when inclusion of minority candidates becomes technically feasible. (2) Tests must be administered and scored under controlled and standardized conditions, with proper safeguards to protect the security of test scores and to insure that scores do not enter into any judgments of employee adequacy that are to be used as criterion measures. Copies of tests and test manuals, includ ing instructions for administration, scoring, and interpretation of test results, that are privately developed and/or are not available through normal commercial channels must be included as a part of the validation evidence. (3) The work behaviors or other cri teria of employee adequacy which the test is intended to predict or identify must be fully described; and. addition ally, in the case of rating techniques, the appraisal form(s) and instructions to the rater(s) must be Included as a part of the validation evidence. Such criteria may include measures other than actual work proficiency, such as training time, supervisory ratings, regularity of attend ance and tenure. Whatever criteria are used they must represent major or critical work behaviors as revealed by careful job analyses. (4) In view of the possibility of bias inherent in subjective evaluations, su pervisors' prejudice, as, when, as new carefully developed, and the ratings should be closely examined for evidence of bias. In addition, minorities might obtain unfairly low performance crite rion scores for reasons other than su pervisory rating techniques should be employees, they have had less opportu nity to learn job skills. The general point is that all criteria need to be examined to insure freedom from factors which would unfairly depress the scores of minority groups. (5) Differential validity. Data must be generated and results separately reported for minority and nonminority groups wherever technically feasible. Where a minority group is sufficiently large to constitute an identifiable factor in the local labor market, hut validation data have not been developed and presented separately for that group, evidence of satisfactory validity based on other groups will be regarded as only provi sional compliance with these guidelines pending separate validation of the test for the minority group in question. (See § 1607.9). A test which is differentially valid may be used in groups for which it is valid but not for those in which it is not valid. In-this- regard, where a test is valid for two groups but one group characteristically obtains higher test scores than th e . other without a cor responding difference in job performance, cutoff scores must be set so as to predict the same probability of job success in both groups. (c) In assessing the utility of a test the following considerations will be ap plicable: (1) The relationship between the test and at least one relevant criterion must be statistically significant. This ordi narily means that the relationship should be sufficiently high as to have a prob ability of no more than 1 to 20 to have occurred by chance. However, the use of a single test as the sole selection device will be scrutinized closely when that test is valid against only one component of job performance. (2) In addition to statistical signifi cance, the relationship between the test and criterion should have practical sig nificance. The magnitude of the rela tionship needed for practical signifi cance or usefulness is affected by sev eral factors, inciuding: (i) The larger the proportion of ap plicants who are hired for or placed on the job, the higher the relationship needs to be in order to be practically useful. Conversely, a relatively low relationship may prove useful when proportionately few job vacancies are available; (ii) The larger the proportion of ap plicants who become satisfactory em ployees when not selected on the basis of the test, the higher the relationship needs to be between the test and a cri terion of job success for the test to be practically useful. Conversely, a relatively low relationship may prove useful when proportionately few applicants turn out to be satisfactory; (iii) The smaller the economic and human risks involved in hiring an un qualified applicant relative to the risks entailed in rejecting a qualified appli cant, the greater the relationship needs to be in order to be practically useful. Conversely, a relatively low relationship may prove useful when the former risks are relatively high. § 1607.6 Presentation of validity evi dence. The presentation of the results of a validation study must include graphical and statistical representations of the re lationships between the test and the cri teria, permitting Judgments of the test’s utility in making predictions of future work behavior. (See § 1607.5(c) concern ing assessing utility of a test.) Average scores for all tests and criteria must be reported for all relevant subgroups, in cluding minority and nonminority groups where differential validation is required. Whenever statistical adjustments are 4a made in validity results for less than per fect reliability or for restriction of score range in the test or the criterion, or both, the supporting evidence from the valida tion study must be presented in detail. Furthermore, for each test that is to be established or continued as an opera tional employee selection instrument, as a result of the validation study, the minimum acceptable cutoff (passing) score on the test must be reported. It is expected that each operational cutoff score will be reasonable and consistent with normal expectations of proficiency within the w ort force or group on which the study was conducted. § 1 607 .7 Use o f other validity studies. In cases where the validity of a test cannot be determined pursuant to § 1607.4 and § 1607.5- (e.g., the number of subjects is less than that required for a technically adequate validation study, or an appropriate criterion measure cannot be developed), evidence from validity studies conducted in other organizations, such as that reported in test manuals and professional literature, may be consid ered acceptable when: (a) The studies pertain to jobs which are comparable (Le., have basically the same task ele ments) , and (b) there are no major dif ferences in contextual variables or sample composition which are likely to significantly affect validity. Any person citing evidence from other validity studies as evidence of test validity for his own jobs must substantiate in detail job comparability and must demonstrate the absence of contextual or sample differ ences cited in paragraphs (a) and (b) of this section. § 1607.8 Assumption of validity. (a) Under no circumstances will the general reputation of a test, its author or its publisher, or casual reports of test utility be accepted in lieu of evidence of validity. Specifically ruled out are: as sumptions of validity based on test names or descriptive labels; all forms of pro motional literature: data bearing on the frequency of a test's usage; testimonial statements of sellers, users, or consul tants; and other nonempirical or anec dotal accounts of testing practices or testing outcomes. (b) Although professional supervision of testing activities may help greatly to insure technically sound and nondis- criminatory test usage, such involvement alone shall not be regarded as constitut ing satisfactory evidence of test validity. § 1607.9 Continued use o f tests. Under certain conditions, a person may be permitted to continue the use of a test which is not at the moment fully supported by the required evidence of validity. If, for example, determination of criterion-related validity In a specific setting is practicable and required but not yet obtained, the use of the test may continue: Provided: (a) The person can cite substantial evidence of validity as described in § 1607.7 (a) and (b ); and (b) he has In progress validation pro cedures which are designed to produce. within a reasonable time, the additional data required. It is expected also that the person may have to alter or suspend test cutoff scores so that score ranges broad enough to permit the identification of criterion-related validity will be obtained. § 1607.10 Employment agencies and employment services. (a) An employment service, including private employment agencies. State em ployment agencies, and the U.S. Training and Employment Service, as defined in section 701(c), shall not make applicant or employee appraisals or referrals based on the results obtained from any psycho logical test or other selection standard not validated in accordance with these guidelines. (D) An employment agency or sendee which is requested by an employer or union to devise a testing program is required to follow the standards for test validation as set forth in these guide lines. An employment service is not relieved of its obligation herein because the test user did not request such valida tion or has requested the use of some lesser standard than is provided in these guidelines. (c) Where an employment agency or service is requested only to administer a testing program which has been else where devised, the employment agency or service shall request evidence of vali dation, as described in the guidelines in this part, before it administers the test ing program and/or makes referral pur suant to the test results. The employment agency must furnish on request such evidence of validation. An employment agency or service will be expected to refuse to administer a test where the employer or union does not supply satis factory evidence of validation. Reliance by the test user on the reputation of the test, its author, or the name of the test shall not be deemed sufficient evidence of validity (see § 1607.8(a)) . An employ ment agency or service may administer a testing program where the evidence of validity comports with the standards provided in § 1607.7. § 1607.11 Disparate treatment. The principle of disparate or unequal treatment must be distinguished from the concepts of test validation. A test or other employee selection standard— even though validated against job per formance in accordance with the guide lines in this part— cannot be imposed upon any individual or class protected by title VH where other employees, applicants or members have not been subjected to that standard. Disparate treatment, for example, occurs where members of a minority or sex group have been denied the same employment, pro motion, transfer or membership oppor tunities as have been made available to other employees or applicants. Those employees or applicants who have been denied equal treatment, because of prior discriminatory practices or policies, must 5a at least be afforded the same opportu nities as had existed for other employees or applicants during the period of dis crimination. Thus, no new test or other employee selection standard can be im posed upon a class of individuals pro tected by title VH who, but for prior discrimination, would have been granted the opportunity to qualify under less stringent selection standards previously in force. § 1607.12 Retesting. Employers, unions, and employment agencies should provide an. opportunity for retesting and reconsideration to earlier “failure” candidates who have availed themselves of more training or experience. In particular, if any appli cant or employee during the course of an interview or other employment pro cedure claims more education or experi ence. that individual should be retested. §1607.13 Other selection techniques. Selection techniques other than tests, as defined in 5 1607.2, may be improperly used so as to have the effect of discrim inating against minority groups. Such techniques include, but are not restricted to, unscored or casual interviews and un scored application forms. Where there are data suggesting employment discrim ination, the person may be called upon to present evidence concerning the validity of his unscored procedures as weE as of any tests which may be used, the evidence of validity being of the same types referred to in 3 § 1607.4 and 1607.5. Data suggesting the possibility of dis crimination exist, for example, when there are differential' rates of applicant rejection from various minority and nonminority or sex groups for the same job or group of jobs or when there are disproportionate representations of mi. nority and nonminority or sex groups among present employees in different types of jobs. If the person is unable or unwilling to perform such validation studies, he has the option of adjusting employment procedures so as to elimi nate the conditions suggestive of em ployment discrimination. § 1607.14 Affirmative action. Nothing in these guidelines shaE be interpreted as diminishing a person's ob- Egation under both title VH and Execu tive Order 11246 as amended by Execu tive Order 1^375 to undertake affirmative action to ensure that applicants or em ployees are treated without regard to race, color, religion, sex, or national origin. SpecificaEy, the use of tests which have been validated pursuant to these guidelines does not relieve employers, unions or employment agencies of f-.boi- obligations to take positive action in af fording employment and training to members of classes protected by title VH. The Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. §§1607.1 et seq., provide in pertinent part: G eneral Pkenclpces- Sncnow 1_ Statem ent o f ?M piae.—A. Need fa r ■uitxform iiy —Issuing -agencies. The Federal government's need far a uniform set of principles an the ques tion o f the iwp of tests and other selec tion procedures has long been recog nized. The Equal Employment Oppor tunity Commission,, the Civil Service Commission, the Department of Labor, and the Department of Justice jointly have adopted these uniform guidelines to meet that need, and to apply the same, principles- to the Fed eral Government as are applied to other employers. * * * C. Relation to prior guideline*. These guidelines are based upon ap d supersede previously issued guidelines -on employee selection procedures. These guidelines have been built upon court decisions, the previously issued guidelines of the agencies, and the practical experience of the agencies, as well as the standards of the psycho logical profession. These guidelines are intended to be consistent with ex isting law. TaCSHHCAIL S ta rba m s Sec. 14. Technical standards far va- liditg studies! The following mi ninmm standards, as applicable, should be met in conducting a validity study. Noth ing in thesE guidelines is intended to preclude the development and use of other professionally acceptable tech niques with respect to validation of se lection procedures. Where it is not technically feasible for a user to con duct a validity study, the user has the obligation otherwise to comply with these guideffnes. See sections 6 and 7 above. A. Validity studies should he based on review of information about the job. Any validity study should be based upon a review of information about the job for which the selection procedure is to be used; The review should include a job analysis except as provided in section 14B(3) below with respect to - criterion-related validity. Any method of job analysis may be used if it provides the information re quired for the specific validation strat egy used. B. Technical standards for criterion- related validity studies.—(1) Technical, feasibility. Users choosmg to validate a selection procedure by a criterion-re lated validity strategy should deter mine whether it is technically feasible (as defined in section 151 to conduct such a study in the particular employ ment context. The determination oi the number of persons necessary to permit the conduct of a meaningful criterion-related study should be made by~the user on the basis of all relevant information concerning the selection ■procedure, the potential sample and the employment situation. Where ap propriate, jobs with substantially the same major work behaviors may be grouped together for validity studies, in order to obtain an adequate sample. These guidelines do not require a user to hire or promote persons for the purpose of making it possible to con duct a criterion-related study. (2) Analysis of the job. There should be a review of job information to de termine measures of work behavioris) or performance that are relevant to the job or group of jobs in question. These measures or criteria are rele vant to the extent that they represent critical or important job duties, work behaviors or work outcomes as devel oped from the'review of job informa tion. The possibility of bias should be considered both in selection of the-cri terion measures and their application. In view of the possibility of bias in subjective evaluations, supervisory rating techniques and instructions to raters should be carefully developed. All criterion measures and the meth ods for gathering data need to be ex amined for freedom from factors which would unfairly alter scores of members of any group. The relevance of criteria' and their freedom from bias axe of particular concern when there are significant differences in measures of job performance for different groups. (3) Criterion measures. Proper safe guards should be taken to insure that scores on selection procedures do not enter into any judgments of employee adequacy that are to be used as crite rion measures. Whatever criteria are used should represent important or critical work behavioris) or work out comes. Certain criteria may be used without a full job analysis if the user can show the importance of the crite ria to the particular employment con text. These criteria include but are not limited to production rate, error rate, tardiness, absenteeism, and length of service. A standardized rating of over all work performance may be used where a study of the job shows that it is an appropriate criterion. Where per formance in training is used as a crite rion. success in training should be properly measured and the relevance of the training should be >hown either through a comparsion of the content of the training program with the criti cal or important work behavior(s) of the job(s), or through-a demonstration of the relationship between measures of performance in training and meas ures of job performance. Measures of relative success in training include but are not limited to instructor evalua tions, performance samples, or tests. Criterion measures consisting of paper and pencil tests will be closely re viewed for job relevance. (4) Representativeness of the sample. Whether the study is predictive or concurrent, the sample subjects should insofar as feasible be represent- f ative of the candidates normally avail able in the relevant labor market for the job or group of jobs in question, and should insofar as feasible include the races, sexes, and ethnic groups normally available in the relevant job market. -In determining the represen tativeness of the sample in a concur rent validity study, the 'user should take into account the extent to which the specific knowledges or skills which are the primary focus of the test are those which employees learn on the- job. Where samples are combined or compared, attention should be given to see that such samples are compara ble in terms of the actual job they per form, the length of time on the job where time on the job is likely to affect performance, and other relevant factors likely to affect validity differ ences; or that these factors are includ ed in the design of the study and their effects identified. (5) Statistical relationships. The degree of relationship between selec tion procedure scores and criterion measures should be examined and computed, using professionally accept able statistical procedures. Generally, a selection procedure is considered re lated to the criterion, for the purposes of these guidelines, when the relation ship between performance on the pro cedure and performance on the crite rion measure is statistically significant at the- 0.05 level of significance, which means that it is sufficiently high as to have a probability of no more than one (1) in twenty (20) to have occurred by chance. Absence of a statistically significant relationship between a se lection procedure and job performance should not necessarily discourage 7a other investigations of the validity of that selection procedure. (6) Operational use of selection pro cedures. Users should evaluate each se lection procedure to assure that it is appropriate for operational use, in cluding establishment of cutoff scores or rank ordering. Generally, if other factors reman the same, the greater the magnitude of the relationship (e.g„ coorelation coefficent) between performance on a selection procedure and one or more criteria of perform ance on the job, and the greater the importance and number of aspects of job. performance covered by the crite ria, the more likely it is that the pro cedure will be appropriate for use. Re liance upon a selection procedure which is significantly related to a cri-. terion. measure, but which is based upon a study involving a large number of subjects and has a low correlation coefficient will be subject to close review if it has a large adverse impact. Sole reliance upon a single selection instrument which is related to only one of many job duties or aspects of job performance will also be subject to close review. The appropriateness of a selection procedure is best evaluated in each particular situation and there are no minimum correlation coeffi cients applicable to all employment situations. IrT determining whether a selection procedure is appropriate for operational use the following consider ations should also be taken into ac count:'The degree of adverse impact of the procedure, the availability of other selection procedures of greater or substantially equal validity. (7) Overstatement of validity find ings. Users should avoid reliance upon techniques which tend to overestimate validity findings as a result of capital ization on chance unless an appropri ate safeguard is taken. Reliance upon a few selection procedures or criteria of successful job performance when many selection procedures or criteria of performance have been studied, or the use of optimal statistical weights for selection procedures computed in one sample, are techniques which tend to inflate validity estimates as a result of chance. Use of a large sample is one safeguard: cross-validation is another. (8) Fairness. This section generally nails for studies of unfairness where technically feasible. The concept of fairness or unfairness of selection pro cedures is a developing concept. In ad dition, fairness studies generally re quire substantial numbers of employ ees in the job or group of jobs being studied. For these reasons, the Federal enforcement agencies recognize that the obligation to conduct studies of fairness imposed by the guidelines generally will be upon users or groups of users with a large number of per sons in a a job class, or test developers; and that small users, utilizing their own selection procedures"Will general ly not be obligated, to conduct such studies because it will be technically infeasible for them to do so. (a) Unfairness defined. When mem bers of one race, sex, or ethnic group characteristically obtain lower scores on a selection, procedure than mem bers of another group, and the differ ences in scores are not reflected in dif ferences in a measure of job perform ance, use of the selection procedure may' unfairly deny opportunities to members of the group that obtains the lower scores. (b) Investigation of fairness. Where a selection procedure results in an ad verse impact on a race, sex, o r ethnic group identified in accordance with the classifications set forth in section 4 above and that group is a significant factor in the relevant labor market, the user generally should investigate the possible existence of unfairness for that group if it is' technically feasi ble to do so. The greater the severity of the adverse impact on a group, the greater the need to investigate the. possible existence of unfairness. Where the weight of evidence from other studies shows that the selection procedure predicts fairly for the group in question and for the same or similar jobs, such evidence may be relied on in connection with the selection proce dure at issue. •(c) General considerations in fair ness investigations. Users conducting a study of fairness should review the A.F.A. Standards regarding investiga tion of possible bias in testing. An in vestigation of fairness of a selection procedure depends on both evidence of validity and the manner in which the selection procedure is to be used in a particular employment context. Fair ness of a selection procedure cannot necessarily be specified- in advance without investigating these factors. In vestigation of fairness of a selection procedure in samples where the range of scores on selection procedures or criterion measures is severely restrict ed for any subgroup sample (as com pared to other subgroup samples) may produce misleading evidence of unfair ness. That factor should accordingly be taken into account in conducting such studies and before reliance is placed on the results. (d) When unfairness is shown. If un fairness is demonstrated through a showing that members of a particular group perform better or poorer on the job than their scores on the selection procedure would indicate through comparison with how members of other groups perform, the user may either revise or replace the selection instrument in accordance with these guidelines, or may continue to use the selection instrument operationally with appropriate revisions in its use to assure compatibility between the prob ability of successful job performance and the probability of being selected. (e) Technical feasibility o f fairness studies: In addition to the general con ditions needed for technical feasibility for the conduct erf a criterion-related study (see section 18, below) an inves- - tigation of fairness requires the fol lowing: v'“d3 An adequate sample of persons in each group available for the study to achieve findings of statistical signifi cance. Guidelines do notTequire a user to hire or promote persona on the basis of group classifications for the purpose of making ft possible to con duct a study of fairness; but the user "has the obligation otherwise to comply with these guidelines. CiD The samples for each group should be comparable in terms of the actual job they perform, length of time on the job where time on the job is likely to affect performance, and other relevant factors likely to affect validity differences; or such factors should be included in the design-of the study and their effects identified. (f) Continued use o f selection proce dures when fainness studies not feasi ble. If a study of fairness .should other wise be performed, but is not techni cally feasible, a selection procedure may be used which has otherwise met. the validity standards of these guide lines, unless the teehiacsl infeashtiity resulted from discraatnanory employ ment practices which are demonstrat ed by facts other than past failure to- conforrH with requirements for valida tion of selection procedures. However, when it becomes technically feaahie for the user to perform aatady of fair ness and such a study is otherwise ra-i>pri far. the user should conduct the study of'fairness. 8a