It looks like the school board will hear from both sides after all about the SAS report.
Today's presentation comparing the SAS EVAAS model and the school system's Effectiveness Index was originally only going to be presented by staff, including Asst. Supt. David Holdzkom.
School board member Ron Margiotta said he sent a letter to Chief Academic Officer Donna Hargens complaining about no one from SAS being invited to attend.
But Trent Smith, a SAS spokesman, said Hargens has now extended an invitation to the EVAAS section to speak at the presentation. Among the people present will be William Sanders, the father of the EVAAS system and the senior manager of the division.
The presentation will come between 3-4 p.m. at today's board meeting. Some people are also planning to talk about the report during the 4 p.m. public comment section.
Comments
There is another kink in the armor...
Wed, 10/21/2009 - 15:48 — midtownmomHigh schools do not have the budget to offer geometry to everyone in 9th grade that took algebra 1 in 8th grade. (At least the school I'm familiar with.) So it's almost a mute point to add extra classes in middle school unless they are added to the high school level. I know several families who were furious (can't blame them a bit) when they got their child's schedule this summer and it didn't include geometry and biology (the science that goes along with it). They were told that there wasn't enough budget available to offer enough classes for everyone who qualified. So there's a much bigger problem that needs to be addressed.
I think you will find out
Wed, 10/21/2009 - 18:23 — user12345I think you will find out that is dependant on the high school.
Bigger issue is state
Wed, 10/21/2009 - 17:05 — shank56Bigger issue is state funding (and local supplements ?) . Program and course cuts happened everywhere.
EVAAS
Wed, 10/21/2009 - 08:56 — lferreriI'm glad to see that the WCPSS is going to start using EVAAS. What concerns me is that I'm not sure that they will use the information to make opportunities available to all students on an equal basis. Right now, Algebra-ready students who are Black or Hispanic have only a 40% chance of being enrolled in this class in eighth grade. (And White students have only a 60% chance.) Will this change? I understand that EVAAS will provide explicit information about an individual student's chances of succeeding, but I believe that teachers and guidance counselors have had some knowledge of this for many years based on their observations of students and on the student's EOGs. Yet they have been using other criteria such as the socioeconomic status and the power of the parents to determine placement in Algebra. We need some way to put pressure on the schools to use the EVAAS information for placement.
We also need to find a way
Wed, 10/21/2009 - 13:58 — jenmanWe also need to find a way to ensure that every kid who is qualified to be on the advanced track are actually able to take those classes.
"Right now, Algebra-ready
Wed, 10/21/2009 - 09:20 — user12345"Right now, Algebra-ready students who are Black or Hispanic have only a 40% chance of being enrolled in this class in eighth grade. (And White students have only a 60% chance.) "
You saw that 85 kids are eligible (under SAS) and probably hundreds are taking the Algebra now ... so, EVAAS will actually decrease the number taking the class!
One school
Wed, 10/21/2009 - 12:22 — lferreriSee the discussion below. That was for Black males in one school. I am sure that there is no school in the WCPSS with hundreds of Black males in eighth grade Algebra.
Dr. Holdzkom was not
Tue, 10/20/2009 - 19:40 — Eric_BDr. Holdzkom was not present at the meeting. In fact, I saw him heading for his car in the parking lot as I was heading in to the building at 2:45.
No explanation was given for the 3 month delay in the report and no one claimed that the Effectiveness Index was the same as EVAAS.
Brad McMillan from E&R gave a Powerpoint presentation describing the EI and its uses. Board members asked several questions after his presentation.
After Mr. McMillan was finished, Dr. Sanders presented EVAAS. He had an assistant present that demoed EVAAS and showed several reports to board members. He showed the district and school reports quickly.
He then got into some interesting reports. He showed an "at risk" student report and an advanced student report. The at risk report was made by searching on all 6th graders with less than a 35% probability of passing the 7th grade math EOG. It was powerful for board members to see that students can be identified individually and targeted for appropriate assistance in reaching grade level. For the advanced student query, he searched for 6th graders with a greater than 70% chance of passing the Algebra I EOC. These students can then be placed in advanced courses and given appropriately challenging work.
Dr. Sanders presentation was very good and I believe that board members were impressed. They asked several good questions afterwards. At the end, Donna Hargens mentioned that all principals are being issued login credentials to EVAAS now without needing to request them from Dr. Holdzkom's office. They are looking to use EVAAS in conjunction with PLTs.
Thanks
Tue, 10/20/2009 - 20:12 — lferreriThanks for filling us in. Was there any discussion about the other information in the SAS report showing that qualified students are being kept out of Algebra I in eighth grade and that race seems to play a role in this decision?
You're welcome. No, there
Tue, 10/20/2009 - 20:21 — Eric_BYou're welcome. No, there was no discussion at this board meeting regarding student placement or any of the conclusions of the SAS report. Both Mr. McMillan and Dr. Sanders stuck to describing what each method can do and how it is used.
Dr. Sanders did show a report to board member showing 6th grade black males with a 70% or greater chance of passing the Algebra I EOC. The results returned 84 students that could then be tracked into 8th grade Algebra. He showed how the data can be used for placement purposes, but stayed away from the conclusions in his report.
"Dr. Sanders did show a
Tue, 10/20/2009 - 22:16 — user12345"Dr. Sanders did show a report to board member showing 6th grade black males with a 70% or greater chance of passing the Algebra I EOC. The results returned 84 students that could then be tracked into 8th grade Algebra. "
84 black kids out of 1500 possible! ... I am guessing more than 84 black males were put in Algebra I without any help from SAS ... I can not believe there wide spread discrimination here ... maybe if 1000 black male were ready and 84 were allowed in ... but 84 ready and probably hundreds?? let in .. does not add up ..
That can't be right. That
Tue, 10/20/2009 - 22:35 — klanders65That can't be right. That must be for an individual school. Even if they said district, it must be for a school.
I could be confused. Dr.
Wed, 10/21/2009 - 07:46 — Eric_BI could be confused. Dr. Sanders and his assistant ran at least 7 or 8 different reports, some for the district and some for an individual school. The district, school, and student names were all blanked out and marked as "District X", "School 0", "Student 1", etc., so it is very possible I misunderstood that report.
Its not Dr. Holdzkom--It's
Tue, 10/20/2009 - 20:08 — klanders65Its not Dr. Holdzkom--It's Mister. He is not a Dr.
This sounds great.
Wake Board of Ed Lauded For
Tue, 10/20/2009 - 19:15 — AngelaWWake Board of Ed Lauded For Diversity
http://wake.mync.com/site/wake/news/story/43431/wake-board-of-ed-lauded-for-diversity
A twist for community advocates came in the full board meeting today when school administrators announced that they will be using a SAS-developed data management program to evaluate student achievement.
Members of the Wake Schools Community Alliance argued as recently as September that Wake should take advantage of the free program that is being used by almost every school district in the state. Wake Schools had argued that its own Evaluation and Research Department already provided similar data for use by principals and teachers.
Dr. Bill Sanders, of the SAS EVAAS group, told board members today that Wake's method of adjusting student measurements based on race or socioeconomic status provides skewed results.
"When you start adjusting for that, it's going to make the child from the impoverished household appear to be doing better than they really are," said Sanders.
Wake's Chief Academic Officer Donna Hargens told board members that passwords would be given to all principals to provide confidential access the EVAAS website and information
Sanders vs. Holdzkom
Tue, 10/20/2009 - 18:28 — klanders65How did Holdzkom explain hiding the report for 3 mos, and how with Sanders there did he do trying to say that his Effectiveness Index is the same as EVAAS? I would have loved to have seen that.
Thanks for the update, Mr.
Tue, 10/20/2009 - 10:14 — Eric_BThanks for the update, Mr. Hui. I'm pleased to hear that WCPSS has done the right thing and allowed Dr. Sanders and SAS an opportunity to explain the SAS report in person before the school board.
Sanders v Holdzkom...SAS v WCPSS
Tue, 10/20/2009 - 07:09 — btwalshSo glad both sides will be heard! This should be an enlightening presentation for the 7 CURRENT board members that didn't feel it was necessary to request (or review) the SAS-EVASS report.
Note to media: please attend! Record and widely broadcast the footage and findings so the public can be informed.
Note to new Board: most of the affected WCPSS customers (parents) are working or tending school children at 3-4pm. Please adopt some family friendly meeting times that will enable families to participate.
Will Both Sides Be Heard?
Tue, 10/20/2009 - 20:06 — chaboardI can read the SAS side at my leisure.....but as far as I can tell I have no access at all to the other side of the story. Keung, are there any plans to make the in-house presentation (or other explanatory material) available on the web?
There is really no other
Tue, 10/20/2009 - 20:15 — Eric_BThere is really no other side of the story. Nothing that was presented by WCPSS refuted anything in the SAS report. It simply was an explanation of how the Effectiveness Index is calculated and what it is currently used for.
If you would like, you can read the original WCPSS report that prompted the SAS response in the first place:
http://www.wcpss.net/evaluation-research/reports/2009/0911evaas_index.pdf
WCPSS is not able to produce the kind of student-level reports demonstrated with EVAAS today by using the Effectiveness Index.
Thanks!
Tue, 10/20/2009 - 20:32 — louiselee44Eric -
Thanks so much for the report! I wanted to be at the school board meeting today, but have a child down with the flu and a husband sick with "something".
Question - does anyone know why WCPSS put out that comparison report in the first place? I'm really glad they did, but if it wasn't required or anything, how smart was it to CHOOSE to question SAS - on collecting and reporting data, of all things!!??
Louise I Just Finished Reading The Document Eric
Tue, 10/20/2009 - 21:21 — chaboardlinked to, beginning to end. And there is absolutely NOTHING in there that can reasonably be interpreted as "questioning SAS". In fact it pretty much says the same things the SAS response says in most places - that the two systems correlate highly with each other on pretty much everything but that there are some differences due mainly to either different scope (local vs statewide) or different technique (using individual performance plus program characteristics to predict performance vs using only individual performance).
SAS responded with a spirited and technical argument as to why they feel it is preferable to use only individual performance (though, unlike WCPSS they will "fill" mising data to an extent). The other side - which we have NOT heard in any forum - is the same kind of techincal defense from WCSPSS analysis department of *their* methodology. It's entirely possible that expanding the information going into the model as Wake is doing could lea to a better model....it would be nice to hear from its developers why they feel it might. I have a lot of confidence in the abilities of the people at SAS....but this kind of model building is NOT usually black & white, right or wrong kind of stuff and in any case it's better to judge the technical argument on the data and reasoning rather than accepting an argumentfrom authority.
I just wish someone could get the technica side from the WCPSS analysts so that we could make that judgement. Unfortunately I'm pretty sure that neither the current board or the incoming board gives a damn what the truth of the modelling may be.....this is only gonna get out to us if the media pushes for it.
BTW, Eric....I realized a few days ago that I have a passing acquaintance with someone in the EVAAS department. Unfortunately it's not someone I know well enough to bug about details.... ;)
Thanks!
Wed, 10/21/2009 - 21:29 — louiselee44But I still wonder why WCPSS did the comparison. Was it required for something else?
It wasn't required. They
Wed, 10/21/2009 - 21:36 — klanders65It wasn't required. They were trying to justify not using EVAAS by showing how it was the same as what they already had so they didn't need it. Actually, what they have can only be used by E&R and the folks in the schools had no access to data that was easy to use. So, by keeping them from EVAAS, which put the data in their hands, the power goes out of E&R. Everyone would no longer rely on them to interpret the world for them. This is why they wrote it.
Actually in this case it is a bit black & white
Tue, 10/20/2009 - 22:18 — TrailerParkGirl"...but this kind of model building is NOT usually black & white, right or wrong kind of stuff and in any case it's better to judge the technical argument on the data and reasoning..."
Actually in this case it is a bit black & white. The WCPSS model discounts expected growth for low-income students and discounts it again for higher poverty schools. EVAAS does not do this. As there is a correlation between SES status and race in Wake County, to some degree that means that the WCPSS model on the whole expects lower growth from minority (including black) students than white students. How is expecting lower achievement from one demographic versus another a "technical difference" or a "technical argument"?
One defination given for discrimation is "the process by which two stimuli differing in some aspect are responded to differently."
The differences in EVAAS vs.
Tue, 10/20/2009 - 21:47 — klanders65The differences in EVAAS vs. WCPSS's model are night and day. No one has ever reviewed WCPSS's model. EVAAS has been reviewed by outside firms. They use a lot of data per child to make a prediction, which reduces the error. WCPSS uses one test score per child, which makes the error so huge that even they say you cannot use their model to know anything about individual students.
Besides the fact that you cannot use WCPSS's model to predict for an individual student, it is not a tool for educators to use. EVAAS is a tool that teachers and principals, and counselors can use to generate a list of students who need more help to succeed, or who should be in advanced classes, or to find what subjects their school is weak in, etc.
WCPSS' s model is not a tool. The principals can only know what subjects they are weak in if E&R runs a report and gives it to them. They can't know anything about students. The model is something E&R can use to do things, but principals don't use it.
Do you understand this difference? They are two entirely different types of things besides being statistically different.
Effectiveness Index is for one purpose. It is for being able to tell if a school did "as expected." And "as expected" is different depending on income of kids and of the schools as a whole.
EVAAS has so many more uses, predictions have much less error, and it is a tool that can be used by staff at schools. In addition to that it does not have any built in bias to have different expectations for students of the same ability but different subgroup members.
And, EVAAS helps us identify students who are already ready to succeed in advanced classes so that we can put them there. WCPSS's model does not.
Most Of The Stuff You Are Saying
Tue, 10/20/2009 - 22:19 — chaboardis right there upfront in the WCPS report. They acknowledge that their EI does not do all the things that EVAAS does. But they are comparing the things that both systems DO do and pointing out - just as the SAS response also did - that the two models on those things give very similiar results.
There is no claim that I can find that EI is intended to predict student performance. The entire comparson WCPSS did between the two (linked b Eric above) is quite explicitly comparing their abilities to classify SCHOOLS. And SAS' response was just that - a *response*. A response to a comparison of the classifications of *schools*. I don't think *either* side has claimed there is any comparison to be made at the individual level - so I'm not sure why you consider that to be the big issue. Care to elaborate?
So where are you getting your information about error bands in the WCPSS model?
error bands
Tue, 10/20/2009 - 23:12 — klanders65They have a report on their website (E&R) that explains Effectiveness Index. It is old. It has nothing to do with SAS. It says that the data cannot be used at the student level because of error. I'll try to find the link. It was old and updated a year ago or so.
I may be wrong but I think the reports and responses were more than just about comparing schools. They were that, too. They were about comparing the models. WCPSS has not been using EVAAS. E&R has been saying they don't need it because they have EI. EVAAS has been available for 3-4 years and hardly anyone in Wake has used it. I had the impression the WCPSS was to show that EVAAS is not needed because EI is just fine. Even if they gave exactly the same results at the school level, which they don't, EVAAS does things that E.I. does not do. I think the SAS Response addressed that by saying EVAAS can be used to identify the kids who are ready for advanced classes.
Sounds like you got it down,
Tue, 10/20/2009 - 21:46 — Eric_BSounds like you got it down, Charlie, although I'm not sure what you mean by "filling in" data. EVAAS uses all available testing data and dampens error by using all data. It doesn't "fill in" data though.
One point I'd like to add... Dr. Sanders commented on the fact that the Effectiveness Index adds corrections in for socioeconomic status and "expects" lower performance for low income students. He said that this methodology was explicitly rejected by the US Department of Education. A low income student and a middle class student with the exact same testing history should be judged the same and expectations for future performance should be the same. We should not expect less from a low income student as the Effectiveness Index does just because they are low income.
BTW, EVAAS uses methodology that is peer reviewed and widely used, including by the NC DPI, while the Effectiveness Index is proprietary to Wake County. There is a reference in the WCPSS report that points to a PDF document describing the methodology and statistics behind EVAAS.
A Few Quick Responses And Off To Bed
Tue, 10/20/2009 - 22:33 — chaboardIt was probably poor paraphrasing on my part but my "filling in" comment was a reference to this bit from the WCPSS doc:
"EVAAS© includes in its models adjustments for students with missing or partially missing historical data"
As for EI methodology being proprietary...that's sort of my point. Until we see it we have no real grounds on which to base a comparative decision. That's why I'd like to actually see a technical reponse from WCPSS to the SAS response.
And finally - it may well be desirable to TREAT a low income student and a middle class student with the same test scores as if they have the same expectation of future success (I certainly would ever dream of arguing against THAT) ........ but at the same time it seems to me to be almost a pure certainty that methematcally speaking they do NOT in actually have the same expecation of success. I'm almost dead certain that given two models with otherwise equivalent sets of predictor variables you would get more accurate predictions if you added in socioeconomic information. Obviously here we are NOT dealing with two otherwise equivalent models....but the point is not irrelevan.
I'll also perenthetically note that the DoE that "explicitly rejected" thst argument was the heavily politicized Bush DoE. As ith ALL "scientific" decisions made by Bush-era agencies it must be treated with a heaping helping of caution and skepticism.
Ok, I'm done for the night.
"Bush-era agencies" Do
Wed, 10/21/2009 - 08:07 — shearertw"Bush-era agencies"
Do you really believe the government agenices turn-over with each presidential administration? We could only be so lucky.....
Let's cover what really happens. When a new president takes over, he may appoint a few figure heads to various agencies. Those figure heads will spend the next 3-4 years fighting with defacto life-long appointed, government wastoids to adopt a new policy. If the president is fortunate enough to be elected for a second term, these government wastoids will go into the "turtle defence" mode, pretending to be implementing the "new" policies while actually just running out the clock for 2-3 more years until the president becomes a lame duck.
If you won't to blame someone, just blame big government.
What makes you almost dead certain?
Wed, 10/21/2009 - 01:26 — TrailerParkGirlWhat makes you almost dead certain?
Would you care to explain how this girl from the trailer park scored in the 99% percentile on reading comprehension on nationalized tests, has a grad degree, and is more successful that most of my middle class HS classmates if mathematically I should not have had the same expectation of future success as my middle class counterparts?
I am just one example among many where I come from where they don't saddle low-income kids with this BS of lower expectations. Sadly, there are too few places like that and too many places like this with too many elitists like you that have been saddling people with lower expectations for too long.
How big of you to be willing to TREAT low-income as if they have the same expectation of success and would never dream of arguing against THAT, even though you think they do NOT.
How would the low income and
Tue, 10/20/2009 - 23:18 — klanders65How would the low income and middle income student get to the same place to start? We are talking about "if they start in the same place..." Lets say they start in the same place in 4th grade. How could that be? How could a low income and middle income student both have learned the same amount? Any minute now, that low income student is going to start learning more slowly? Why hasn't it happened yet for him? With the EVAAS model, if they get to 4th grade having learned the same amount, it assumes these two kids are capable of learning at the same rate. EI assumes if they get to 4th grade having learned the same amount, the low income student is about to slow down his learning soon. So, we should start treating him like a slow learner now because although he has learned at the same rate as the middle income kid, he is about to slow down.
Eric... so now the
Tue, 10/20/2009 - 22:21 — user12345Eric... so now the principals have a sign on to both systems and can get all the reports they want ... are we done with this statistical software comparison now?
Principals don't sign on to
Tue, 10/20/2009 - 23:20 — klanders65Principals don't sign on to Effectiveness Index. It is simply a model used on a computer in E&R. No one else can do anything with it. It isn't a system. It does not give reports. EVAAS and Effectiveness Index are two entirely different things. I don't know why they were ever compared. My understanding is that E&R wrote that comparison to have evidence that WCPSS does not need EVAAS. Why did they write it if not for that?
What time do you suggest?
Tue, 10/20/2009 - 07:20 — shank56What time do you think the BOE should meet that is the most family friendly for you?
BAC meetings are in the
Tue, 10/20/2009 - 20:11 — AngelaWBAC meetings are in the evening, why not BoE meetings?
Evenings...
Tue, 10/20/2009 - 20:06 — Bob_SconceEvening meetings allow people with day jobs to serve as school board members. Nearly every school board in the country meets in the evening -- WCPSS only moved it into the day because the school board has historically had a tough time prioritizing where to best use its time and when to delegate.
evenings
Tue, 10/20/2009 - 08:02 — loriacI would vote for evenings - if enough notice, I can usually rearrange my schedule to attend an important evening meeting. During the day - I work, and on some days have kid pickup, so an afternoon mtg is a no-go for me.
Several of the new school
Tue, 10/20/2009 - 08:17 — CaryCurmudgeonSeveral of the new school board members also have day jobs, so I think evenings would work out best for everyone (except staff).
Maybe they could modify
Tue, 10/20/2009 - 09:39 — woodstockMaybe they could modify staff start times by 10 minutes every day to make up for the extra time required in the evening.
Brilliant idea, Woodstock!
Tue, 10/20/2009 - 19:58 — petehsBrilliant idea, Woodstock!
Evening meetings are
Tue, 10/20/2009 - 09:12 — KnightdaleParentEvening meetings are already the norm at the school level. How often do the schools expect parents to show up in the middle of the day for a PTA meeting or open house. I would think it should not be a problem for the administrative staff to follow that same schedule.