Pre-Application Meeting Videos and TranscriptsApril 22, 2002
DR. SWEET: This is the questions portion of our program and I want our resource persons to identify themselves. We have Ron Cartwright, we have Chris Howard, we have Pat Knight, we have Elizabeth Pare and they're going to help field questions that you may have. And there are other colleagues in here, as well, who may be able to help us out. So, any clarifications before we break? Yes. You're going to have to go to the mike, if you would, please, and state your name and your affiliation.
MR. FREEMAN: Garrett Freeman, I'm with a private company called Aptima. Two questions. There have been a number of references to academic organizations applying. I just wanted to confirm that private companies are eligible, okay?
And the second is, there was a reference earlier to submitting IRB materials. Does this imply that your IRB is going to do the review rather than the one we typically use, for example?
DR. SWEET: Your IRB.
MR. FREEMAN: Fine.
DR. SWEET: And we have Ira Pritchard, by the way, who is also quite an expert in this area...
MR. FREEMAN: Great.
DR. SWEET: ...if you need some additional assistance later. Other Q's?
MR. ALBERTINI: John Albertini from the National Technical Institute for the Deaf. My first question is about special populations. Is this open to special populations?
DR. SWEET: Sure. Yes, indeed.
MR. ALBERTINI: Okay. And just to verify the age limit, post secondary...
DR. SWEET: There is no age limit.
MR. ALBERTINI: Okay, fine.
DR. SWEET: There are no parameters on age.
MR. ALBERTINI: Great, terrific. And, finally, you mentioned e-mail. I didn't find your e-mail in here, is that...
DR. SWEET: I believe it's in one of the notices, but I'm going to give it out again.
MR. ALBERTINI: Great, okay.
DR. SWEET: As a matter of fact, all of our e-mail addresses here at ED ends with the same suffix. So, mine is A-N-N-E, anne.sweet@ed.gov. Yes?
MS. ANGLES: I'm Penelope Angles, Educational Testing Service, and I have a very basic question. How are you defining reading comprehension?
DR. SWEET: Well, we also want to hear from you how you're going to define reading comprehension, but I will say to you this, is that our Agency views reading comprehension as a process that's multidimensional, multifaceted, takes into account the reader, the activity, the text, sociocultural context. It's really quite a complex process but, of course each and every person who is making a proposal to us will be using a frame of reference in a theoretical construct frame of his or her own based upon the empirical evidence that perceives your work.
MS. ANGLES: I have just a follow-up.
DR. SWEET: Mm-hmm. (Indicating affirmatively.)
MS. ANGLES: We know there are a number of theories.
DR. SWEET: Right.
MS. ANGLES: Is there one or more particular theories that OERI is particularly interested in receiving proposals developed around those?
DR. SWEET: Valerie, do you have any comment on this before I continue?
DR. REYNA: Sure. I think I've turned on the mike successfully. Yeah. No, we're not interested in particular theories in that sense at all. We are interested in research, in theories that are advanced on the basis of empirical evidence as Anne just said. So, scientifically supported theories.
The idea is you're making a compelling case to reviewers, that's your audience. And so, if you have a theory that is, there's a lot of evidence for and that you're going to say, for example, test a crucial prediction of that scientific theory and then the methods in fact are a valid test of that theory, that would seem to me to be one script, one possibility for a strong application.
If you have a theory that has absolutely no empirical evidence to support it and yet you want to make the case to reviewers that we should spend taxpayers money to do this, that to me is not as strong a case. You know, of all the things that can be supported with very limited taxpayers dollars, why this particular thing? That's the argument you're making.
So, you have to say, well, this is the most significant work, it's the, you know, the most rigorous methods, the most important question that could be asked and this is an excellent way to answer it. That's the kind of case you want to make to reviewers.
So, no, we don't have any particular biases. I, personally, as a reviewer have favorably rated applications that have criticized my own theories. It's just, the question it all boils down to is what's the nature of the evidence and the arguments that are made.
DR. SWEET: Good question, good
response. Others? All questions are good questions. Tom?
MR. ESTES: I have about two, oh, I'm Tom Estes. I am a Professor Emeritus at the University of Virginia and I'm interested in collaborating on this project with both an independent private organization comparable to, but much smaller than ETS, of course, and the School Reform Initiative that's directed by a Regional Service Authority in the State of Florida and with a few of my colleagues with an interest and history of concern in reading research.
My question is, given the consortium, if you will, of individuals interested in this project, who would be most advantageously the project director or, that is to say, the Application Manager? You have a lot of different kinds of people mentioned, but I don't quite know who to put the star next to. Is that a...
DR. SWEET: That's a very interesting question. I mean, although...
MR. ESTES: Does that mean you don't know?
DR. SWEET: No, this is my answer, Tom, and then I'll let Val elaborate because we actually get this question in different kinds of iterations. Of course, we encourage collaboration, but only one partner in the collaboration can be deemed the university or agent of record, if you get my meaning clear. There can only...
MR. ESTES: Sometimes you've mentioned higher education. Does the person have...
MS. KNIGHT: There can only be one...
MR. ESTES: ...to be there?
DR. SWEET: There can only be one agent. Now, as to whether or not the person has to be there I'm going to pull on the wisdom of my OERI colleagues in the room. You've got to go to the mike.
MS. KNIGHT: When there is collaboration, as Anne mentioned, there is, the government needs to deal with one point of contact and that point of contact also is responsible for the physical administration of the grant. And so, it would be whomever you would choose that you think would be in the best situation to provide that kind of relationship with us. Does that help any?
MR. ESTES: So, that can be a person in the private agency?
MS. KNIGHT: Could be.
MR. ESTES: It could be their Budget Director or their...
MS. KNIGHT: Yes.
MR. ESTES: It wouldn't have...no?
MS. KNIGHT: Can I just clarify?
MR. ESTES: Yeah.
MS. KNIGHT: You can have a grantee be in one organization and a project director somewhere else in another organization. They don't have to be in the same place.
MR. ESTES: Oh.
MS. KNIGHT: But the grantee is in charge of the money and is the only organization that can draw down money and there are rules on how that's paid. They can't subgrant, it has to be a contract with and there has to be a legal thing, but the project director and the agent, the grantee, do not have to be in the same organization, but I would encourage you to have a very good understanding of how that's going to work because we have seen...
MR. ESTES: Of course.
MS. KNIGHT: ...problems happen.
MR. ESTES: That's a family matter, right, yes, I know.
MS. KNIGHT: It is and the grantee controls the grant and could substitute a new project director that was qualified because the grant belongs to the grantee, it never belongs to the project director.
MR. ESTES: Good, that's good. This is probably at your height.
DR. REYNA: The person who's the logical leader is the intellectual leader. This is not necessarily the person with the greatest percentage of effort or time on task. Oftentimes it is, but it doesn't have to be at all. So, the person who's the leader is the person who has the scientific or intellectual leadership of the project. This is someone to whom we, as an organization, could look and say, yes, this person, you know, really could be responsible for this project. They have the training, the background, the skills, the experience to be able to manage this project in all its aspects and, in particular, in terms of the research integrity of the project.
If this is someone, for example, who lacks any experience with research and yet they're in the leadership role, reviewers might look askance at that because this person is the ultimate bottom line for quality control. So, again, from a scientific point of view as opposed to the important technical aspects of this, as a contact person I would say it's a person who is the intellectual leader of the project.
DR. SWEET: And what Valerie just said is very important, but I'd also like to underscore what Pat just said. The owner of a grant is the agency that is listed as the prime. So, you really do need to look into how all of that works and if anything went wrong, what would be the consequences.
If you know what I mean, Tom. For instance, if X university gets a grant because an intellectual owner PI applies and is at the university, if that individual should move the grant does not necessarily move with him or her. The university owns the grant and they may wish to keep it and the main PI, of course, it would have to, let me also add another component here. OERI has to approve changes in key personnel. That's just a factor to keep in your mind. Nonetheless should a key personnel move from one institution to another it is not necessarily the case that the grant will follow him or her. It's contingent upon your working out with the university whether or not it would travel with you.
MR. ESTES: And I'll raise my eyebrows. Well, we have a little experience with that, now.
DR. SWEET: Okay
MR. ESTES: Okay, I'm looking for the distinction between the PI and the project director.
DR. SWEET: Okay.
MR. ESTES: You're saying they are the same person, the principal investigator and the project director. Now, I understand the project director...say what? Would the project director be the person fiscally responsible and the...
DR. SWEET: Yes, Pat, help yourself.
MS. KNIGHT: The fiscal question, this is not necessarily so. You might, the project director is responsible and should oversee everything and you would have to work out some way of paying bills and making sure that it worked because you are the project director, but the fiscal agent can be somewhere else. You could have, it depends on how your organization works and that's why they don't have to be in the same organization. And I don't want to minimize who the project director is, but there's a technical thing of paying bills and you have to do that legally. You will go through an audit, there are all kinds of things like that.
So, there has to be someone who is fiscally responsible. Don't ignore that part because you don't want to get in trouble with the federal government for how you've used funds. And the project director is in charge of the overall but, sure, the project director has some oversight.
MR. ESTES: Right.
MS. KNIGHT: But you can work out something and have a budget office, a finance officer, someone like that. I mean, it's an important part, but it doesn't have to, in fact, it very rarely is the same person. You might have initial layers or whatever it is, how you manage a big project. If you have a multimillion dollar project you don't have one person in charge, but you have to have internal controls. That's for your own protection. We probably won't look at it, we'll assume that you have it because if you don't the audit will catch you and you'll be in trouble.
MR. ESTES: I'm trying to make this simple for me. Of course, I'd like to be the principal investigator and have somebody else do all that other work. Whose expertise is in that, like managing budgets and, you know, there could be a manager of a company who is not the CEO of the company. There probably would be, wouldn't there? So, that's...
MS. KNIGHT: That's really up to you to decide.
MR. ESTES: That's strictly, okay.
MS. KNIGHT: You're responsible. If you want to do it you've got to propose something and you've got to have a project management. Again, we're not telling you how to manage your project. We want this to be a good project with a good design, but you have to manage it. And that is your problem and it really is your problem to figure it out.
MR. ESTES: Okay, here's a more simple, a simpler question, Anne. Will there be meetings of participants held for sharing ideas or results as the project should proceed and will we need to build in travel for that sort of thing?
DR. SWEET: That's an excellent question. We're hopeful, but I'm not sure we built that component in. We did not, but we're hopeful to get...
MR. ESTES: That means no...
DR. SWEET: The answer is it is not mandatory, but we're hopeful that we could, of course, speak with you and all PI's.
MR. ESTES: You probably need to stay up there. Well, you see why I'm asking.
DR. SWEET: I do see.
MR. ESTES: It could be expensive if you were in Oregon and...
DR. SWEET: Absolutely and we did not ask that we build that into your, that you build that into your budget.
DR. REYNA: I would expect that because this is a research application that folks who would be engaged in this would be planning on attending scholarly or scientific conferences where your peers who are also doing research on reading comprehension, those supported by us and those supported by other agencies, would be present.
So, that is certainly a logical thing to include in your travel budget. So, the leading national or international meetings at which the cutting edge of the scientific research is being presented in reading comprehension, those would be the kinds of things one might see in a travel budget.
DR. SWEET: Okay, questions from anyone else before we break? One more.
MS. FORD: Oh, I'm actually tall enough. I'm Lynette Ford, I'm representing a cultural institution in Cleveland, Ohio called A Cultural Exchange and I'm here today with two of my colleagues, Deborah McCann and Lloyd McCann. We were so excited about this RFP and the fact that you talk about wanting to change instructional practice and, yet, after the comments of this morning, particularly with our interest being in the sociocultural context that motivates children to increase in their reading comprehension, I'm questioning this idea that your research has to be based on current empirical data when you're talking about changing current practices that, of course, may be based on the current empirical data. Can you just clarify that for us, please? And I'll sit as you respond.
DR. SWEET: I'm going to do part one and just point out and remind ourselves that the accretion of knowledge is incremental. And so, that in order to make advances forward we always have to look a little bit back because we're building a pyramid of knowledge. Where's Val?
DR. REYNA: Right here. She was fleeing out the door, no. I'm not really sure if I understand the question and I was doing something else, answering another.
DR. SWEET: I think the flavor of her question is, the speaker said that if we're seeking to change practice, then novel ideas that are put on the table may not necessarily have a long trail upon which to build.
DR. REYNA: Oh, okay. Okay, yes. Innovation is actually highly prized by reviewers. You'd be amazed how excited reviewers get about any innovative ideas. They crave them and they will really give you the benefit of the doubt. However, innovation doesn't mean totally without foundation. You have to make an argument, a series of steps that lead to your innovative idea.
If it's just supported by nothing, no prior evidence, no argument that leads logically that this might be something that would be interesting to study, then it's going to be a very hard bet for a reviewer to invest in because, remember, that's the judgment the reviewer is making. The reviewer is trying to pick what's going to pay off. This is a good bet for you, too, for your career. You're basically, you know, and you have an alliance with the reviewer. They want, really would like you to be successful and it would be good if you were successful.
So, the idea is, is this a line of research that's going to pay off? But I think innovation is to be highly encouraged if you can make an argument why this innovation, why in this particular way and why this idea and not all the other ideas that could be done.
MS. FORD: Thank you.
DR. SWEET: Sure. We have time for just one more question and let me just say this. We know that there will be more questions and we will continue this Q and A after Dr. Trabasso does his presentation at 2:30. So, one more question. Who's it going to be? Go.
MS. MacNAMARA: Danielle MacNamara, Old Dominion University.
DR. SWEET: Hi, Danielle.
MS. MacNAMARA: I have a question about what the stress is and whether or not there is a stress in terms of intervention research or classroom based research. That is, I think historically and maybe culturally in these type of programs there is somewhat of a stress on having immediate application in the classroom, but I'm hearing from you that perhaps more theoretical laboratory empirical research may be acceptable. That is, how important is it that we include some classroom based research, how immediately translatable to practice does it have to be?
DR. SWEET: Well, I believe that this program is supporting a range of different kinds of research. So, some of the research that's going to be proposed will be quite theoretical and let me say more basic than other kinds of research. So, there will be a full spectrum that we expect to support. And one of the reasons that we're underscoring this fact is that in prior, the most recent prior competitions in the area of reading OERI went through a period of time where it sort of kind of truncated the basic research and we want to make it very clear that we are re-initiating the incorporation of basic research into this program.
MS. MacNAMARA: Okay, that's the message I was hearing.
DR. REYNA: And that's very well put, Anne. I would definitely make an argument about educational significance, though. So, if you're going to do something that's more toward the basic end of the spectrum, connect the dots for the reviewer. Do not leave it up to the imagination of the reviewer to figure out why this is significant, make that argument as explicit in a pithy way as you possibly can.
DR. SWEET: Okay, we're going to break now. We've only got five minutes. So, you may be excused for five and then be back because we're all really anxious to hear Tom do the presentation. Thanks.
(WHEREUPON, a short recess was taken.)
Pre-Application Meeting Videos and Transcripts
This page last modified May 30, 2002 (jer)