USAID - Responses to the New AEF Form

Comments from the Field on the New AEF System
Posted 6/8/2005

1. Unlike the old system it leaves the supervised employee incredibly vulnerable to the writing skills of the supervisor. Under the old system Missions consciously organized their committees to include at least one good writer. This year I sat on a committee where the supervisor wanted to do well by his employee but the supervisor had limited writing skills. Fortunately the supervisor recognized his writing weakness and the committee was able to clean up the AEF - up to a point. However, after a couple of go-rounds we reached the point of defensiveness. I think the employee may have suffered as a result - although I think that was not the intent of the supervisor. Now imagine a situation where the poor writer was a Mission Director or a Deputy (this is a problem that is more common than I think folks realize.) In the case above, the committee was able to voice its views fairly easily because the supervisor was mid-level. My guess is that committees are going to be much less prepared to actively voice their views when the rater is senior - even when the rater is trying to do well by the supervised employee.

2. Re the new AEF format: I had been writing an AEF this year, I would have been inclined to present the narrative in four main paragraphs, each portraying performance by skill area - with work objective and performance measure achievement built around that format. As it is, virtually every AEF I saw as an appraisal committee member was presented in however many main paragraphs as there were work objectives, each portraying achievement of work objectives and performance measures - with skill area performance sprinkled variously throughout. It will be interesting to find out which pattern prevailed overall. And maybe other patterns will emerge also. The reason I prefer skill area ordering of the narrative is that the skill areas will presumably endure from year to year and progressive performance against skill standards should tell us more about an employee's readiness for promotion than the achievement of miscellaneous and often haphazard/artificial work objectives.

3. Per the request for comments on the new AEF: I thought the new AEF format was wonderful. To eliminate the narrative on work objectives and replace that with checked boxes was a huge time saver without, I bet, impeding the ability of the promotion panels to effectively evaluate candidates. Don't we all meet our work objectives? Why spend time writing a narrative on it. And if someone does not actually meet their work objective it could easily be addressed in the one narrative. I thought this was a hugely positive change. For the future I think clearer guidance on how the one narrative should be structured would be helpful. For example, I thought that the narrative should focus on each skill in the skills matrix and the paragraphs should be labeled accordingly. I also thought that the narrative should focus on examples not written about in the work objectives - get the most bang for the buck by having great things in the work objectives and more, but different, great things in the narrative. But, I think others felt that the narrative should be structured by work objective with the skills being highlighted within the paragraphs. In short, it would be interesting to know what the panels thought was the most effective structure of the narrative.

4. The AEF process doesn't get any easier (doubt it ever will). It comes down to how good a writer the rater is. Space, space, space. As a rater and AC member, it was very hard to get all 4 skills, sub-skills, work objectives and performance measures mentioned in the 1 page, plus say anything helpful to the panel. I am just glad I am not on the panels this year. It would be helpful to have some shorthand for the skill areas. I don't imagine many AEFs rate the employee on "Information Gathering Analysis, and problem Solving - Includes Knowledge Management" because they can't waste half a line on a sub-skill title. I recommend a shorthand code. Eg. 3.3 for the above. Many of us used some shorthand for WOs and PMs, but it would be easier on the panels to standardize them. Another annoyance. In the 21 st century, is there no better form manager which allows editing within the macros? I would rather do without the macros than have to export the section to Work, paste special, check the spelling and grammar and then import it back into the form. Please.

5. The shape of the form isn't as important as the skill of the writer. And writers' skills are hugely constrained by requirements to 'tick-the-boxes', to cover prescribed topics by using prescribed phrases and headings. The reams of 'guidance' (it's of variable usefulness) should be accompanied by 'writers' workshops' - practica for AEF writers (also or separately for the so-called 'suicide box' writers?) USAID generally prefers to compensate for human deficiencies by refining rules and forms; better to strengthen human capacity directly.

6. On the AEFs, the one page format is a HUGE improvement, as is the limitation of 3 WOs and 2 PMs each. One page is plenty of space and avoids repetition or reaching. The midterm box is also a good innovation, as is the AC signatures being optional on the WO and mid-term boxes. One comment is that the Role in the Organization is quite long. I wonder if some Missions are more creative than mine and use it to help make the case - we just used boilerplate that is so lame I can't believe the Boards will read it carefully or at all.

7. From an AC Perspective:

1. It was a much easier task to review each AEF as it was shorter. The usually prolonged discussions on whether or not someone should be recommended for promotion or whether the text supports one position or another was gone, a welcome change.

2. Given the fewer work objectives and only one page to talk about the employees skills, Section 4, Role in the Organization, became even more important part of the AEF and still raters and employees wrote it without much care or life. It is an important opportunity to talk bout the much broader roles and responsibilities than the work objectives are intended to capture.

3. I saw one rater use Section 6 effectively to incorporate AIF input at a mid-cycle that would have otherwise not been captured in a work objective. I recommend that raters and employees use Section 6 not to dryly record changes to work objectives but rather to talk about the successes the employee has made at the mid-cycle point. This can be particularly useful, given the fewness of work objectives, when one work objective must be dropped to make way for another one that captures an important opportunity not foreseen in the first 45 days of the rating period.

4. Raters for the most part are not using the Skills Feedback Worksheet effectively and rarely provide it to the AC with the AEF. When a SFW is done it appears to be an afterthought by the rater with simple check off ratings. Raters should be encouraged to use the SFW as a drafting tool for the AEF. In fact, I would suggest changes to the SFW to allow check off boxes associated with each skill and sub-skill that indicate whether 360º feedback supported a particular skill or sub-skill without reference to source and which sub-skill(s) under each skill area the rater has decided to comment on in the Section 7. Then I would encourage the rater to distill from 360º sources, AIFs and their own observations in the comment section. From the comments they can distill what they then want to focus on in the AEF, Section 7. With the SFW in hand, the AC can quickly see which sub-skills that rater will focus on, whether there was 360 input for that sub-skill, and get insights further insights into the employees performance that might not have been captures in Section 7.

5. The most troubling aspect of serving on an AC is recognizing that not every rater is an effective writer, that can tell a story that captures the readers attention and describes the full impact of an employees work while effectively conveying substantive examples of demonstrated skills. My initial response as a member of the AC is to want to ask for an electronic version of the AEF and do some heavy editing. With only one page to capture an employee's accomplishments and discuss them in terms of skills evidenced, some raters simply cannot do that as succinctly and effectively as others which may further disadvantage the employees during performance board reviews.

6. The EEP Guidebook emphasizes the importance of clearly stating the skill and sub-skills being discussed in Section 7. Some raters took to writing paragraphs dedicated to each skill area with the skill area in capitals. Others raters structured Section 7 around work objectives and other accomplishments and embedded references to skill areas throughout the text (making it a little more challenging for an AC to make sure all skills and subs-skills were addressed). Still others combined the two with the first couple paragraphs focused on work objectives referencing key skills and sub-skills and the remaining four paragraphs each addressing a specific skill area with examples drawn from work objectives and other work the employee did during the rating period. On the whole, each approach had its merits but in most instances the "story" being told was difficult to convey. The use of a consistent metaphor to convey the story was practically non-existent. This would have made it easier for the performance boards to follow the thread and help them understand the employee's contributions and demonstrated skills.

7. The biggest cause of a weak AEF remains poorly written work objectives. Few employees asked ACs to review their work objectives this year in order to help improve them. Work objectives rarely convey the impact and importance of the work and are heavily skewed to process. For many legal, contracting, financial and administrative professionals they assert that their jobs are process oriented or that they cannot know sufficiently far in advance what the important work will be that they will be called upon to perform during the rating period. This is especially problematic for regional service providers serving multiple client missions. To correct that, you almost have to rewrite one or more work objectives in the last 2 months of the rating period to capture work accomplished that could not have been foreseen at the beginning or mid-cycle of the rating period.

a. We decided to take a new approach this year in helping raters write better work objectives. The Front Office has drafted the major objectives it is setting for the mission for the rating period. They are written as outcome "so-what" statements. There are 14 objectives on the current list. Raters are encouraged to pick at least one of these work objectives for each subordinate and tailor them to reflect the individuals role in achieving them. The rater is free to choose other work objectives for their office that also express the skill areas the employee wants to develop or demonstrate. The senior management team of office directors has been asked to ensure that all mission objectives are addressed in one manner or another in someone's work objectives whether they be a USDH, USPSC, or a FSNPSC.

8. Confusion existed on how to interpret the guidance "Highlight one sub-skill area from the next higher level for the employee to develop further." Some were worried that written one way may suggest that the person is working at the next higher level and therefore is not ready for promotions. Most raters took the safe approach to merely say that the employee was already performing at the next higher level for a particular sub-skill without providing a context in which that sub-skill will or is being expressed. Further guidance from M/HR is needed on this requirement.

8. I would like to bring to your attention issues with three sections of this year's AEF form that already have caused confusion as AEFs were written, and that may result in problems with AEF reviews by the panels.

1. Section 6, Formal Mid-Point Progress Review: This section has been interpreted by some to involve solely information related to changes in work objectives and/or performance measures, and by others to allow additional space to comment on an employee's performance. As a result, some employees will benefit by the space given to additional comments, and others will not benefit if their raters and ACs made the more narrow interpretation of the section. HR guidance was not clear.

2. Section 7, Assessment of Performance, Skills and Potential: Some raters chose to write this section using skill areas as a basis (i.e., paragraphs organized around the skill areas with performance examples included), and others chose to write the section using the work objectives as a basis (i.e., paragraphs organized around the work objectives with skill areas highlighted). There probably is no sole right way to prepare this section, but panel members may subconsciously prefer one over the other, and this may disadvantage those employees whose AEFs were written in the less preferred way. I don't think that this is as serious a concern as #1 above and #3 below, but it needs to be raised.

3. Section 8, Appraisal Committee Member Comments: Some ACs have interpreted the content of this section to be solely the documentation of issues the AC had with the rater which could not be resolved and warranted mention in the AEF, while other ACs have used this section to provide (usually) positive feedback by the AC about the employee. Those who followed the first interpretation ordinarily have left the section blank because there were not issues that had to be raised. Thus, some employees will receive favorable, explicit AC support and others will not - but not because the ACs did not want to support them. I believe that the confusion over this section will be the most problematic for employees.

4. Section 7, requirement for area of development: According to original HR guidance, raters were to include in Section 7 discussion about an area for development - a subskill area at the next higher level which the employee could further develop. However, through an email issued by David Eckerson, HR changed this guidance to say that raters could instead talk about a subskill area at the next higher level which the employee already was achieving. My experience in ANE is that not all ACs heard about this revised guidance. As a result, some employees will have in their AEFs an area for development, and others will have an area where already they are performing at a higher level. Probably all employees would have the latter if raters understood revised HR guidance. Again, because of a confused situation, some employees may be disadvantaged.

9. Regarding the new AEF, our Mission required that AEFs identify a skill area/area for improvement that the employee would work on during the year ahead. No big deal, but it seems too bad to have a good review end with an area for improvement. Is this required and can AFSA explore the possibility of removing this requirement from future AEFs? Overall I had a good experience with the new form and think that it is better than the previous one.

10. In response to your request for feedback on the AEF process, I can offer the following based on my experience as a rater - as well as a ratee - and with the added background of having served on the 2 to 1 promotion panel last summer.

- Overall: The short form seems, from the standpoint of time requirements, to be an improvement. It certainly was quicker to write and less time consuming for everyone involved.

- Using check boxes for all skill areas met and all WOs met was a good idea, saving space and time.

- As someone who has served on the panel, I am concerned that this form will make it a great deal more difficult to successfully identify those who are truly most deserving of promotion. The short form makes it much easier to elide over weaknesses and at the same time much less clear to a reader whether something - like a key skill area - is being left out because its a deficiency or being left out because there just was not enough space to talk about it.

- Prohibiting raters and ACs from making promotion recommendations makes our life a lot easier, but - speaking again as a former panel member - will make the panel's work a lot harder. The fact that we could not make statements about promotion relieved us of a big burden. It meant that we did not have to distress several of our weaker performers by explicitly denying them a promotion recommendation . However, these promotion recommendations from a panel perspective are a very useful sorting device. The claim often stated that "everyone gets recommended for promotion" is not true, and furthermore not insightful. Promotion recommendations normally are numerous but not universal. Therefore when an AEF lacks a recommendation that is a clear signal to the panel that the rater/AC definitely believe the employee is not suitable for promotion. Sometimes this is because the person is performing more poorly than the overall narrative acknowledges and sometimes it's because the employee, though excellent, is so new in grade that promotion would be premature.   In either case, it clarifies the rater and AC's real views. Even more important, promotion statements are not all the same and the manner and enthusiasm with which a promotion recommendation is phrased also gives the panel important clues about how really excellent the employee's performance was.

- Interpersonal skills have essentially disappeared from the new skill matrix. This is a mistake. Once I started really working with the new skill matrix (which one has to do when drafting evaluations), I was surprised and somewhat alarmed to find that interpersonal skills have essentially been deleted. It had initially been my impression that the new matrix simply collapsed the old one down into four categories but did not substantively change it. Working with the matrix, I realized that's not true. What's gone is the human element. This came home to me when I realized that one of my rated employees, who is technically proficient and hardworking but weak in human interactions to the point where nearly her entire staff had complained about her in 360, actually scored very well on the matrix. Although clearly deficient as a manager of people, the matrix was written in such a way that her deficiencies did not correspond to any subskill that she was required to meet. Her staff, for example, had complained that at meetings she would detail at length her views, positions and plans - and was not open to hearing anyone else's, that when they ventured to share their opinions, she would listen politely but ignore whatever they said, that even when unanimously opposed she would adhere stubbornly to her own opinion and treated their disagreement with her on any subject as a problem to be solved by repeating her own views more forcefully. From a common sense perspective, this is a serious problem and shows weaknesses in both communication and interpersonal skills. From the new skill matrix perspective, this was no problem at all! The pertinent subskill said something like "communicates views clearly". Well! This person does communicate views clearly. No one said otherwise. The fact that this clear communication is counterproductive, off-putting to the listeners or so relentlessly, stubbornly clear as to be perceived as closed-mindedness and unwillingness to listen to other opinions was not captured in the matrix.

- The instructions were too scattered, too disorganized and insufficiently clear. The AEF instructions and guidance are in innumerable different places - in the forms themselves, in the ADS, in the PowerPoint slide show that HR distributed, in the employee evaluation guidebook. The information in each of the above sources is overlapping but not the same. Much of the info is duplicative (which encourages people to not read it all) but some different things appear in each item. You had to read all of everything in order to get a clear picture. This should not be. All the information - ALL of it - should be in one place. It should not be necessary to access numerous different documents in different places nor read large quantities of redundant material to know what is required.

- Poor instructions re one skill at the next higher level. Despite (or perhaps in a way because of) the numerous overlapping documents - several of which are rather voluminous in length - certain extremely important and basic questions were not answered in the needed clear and unambiguous manner. The requirement to address one skill at the higher level was buried in the ADS. As far as I can recall, it appeared nowhere else. We remembered hearing that such a thing was to be included and were actually looking for the language and could not find it, even knowing that it was there. That's how buried the thing was. Worse yet - the explanation of what was intended by this requirement was vague and unclear. No one understood what it meant and everyone had a different interpretation. From the panel perspective, this is a disaster because some raters will have used it as an area of improvement, some will have used it as an area of excellence to be more broadly deployed on behalf of the agency, some will have failed to notice the requirement or simply ignored it. This damages the comparability of AEFs and may disadvantage individual employees. If the thing is supposed to be an area of improvement (and by the way, now that I've sat on the panels, I support the inclusion of an area for improvement as those items were remarkably enlightening), then it should state that it's an area of improvement.  If it's supposed to be an area of excellence - a statement of the employee's single strongest skill - that might be enlightening also.   Either way, be explicit and clear. If AFSA and or Management are unwilling to be clear - get rid of it. Under no circumstances, should the present formulation be continued as that was a mess.

-Lack of clarity regarding what it was mandatory to cover. The instructions also failed to be crystal clear about whether we were required to address every skill area explicitly and whether we are required to address explicitly every WO, every performance measure (or just one performance measure under each WO). I eventually found a statement about this in the PowerPoint slide show. These are critically important questions when one is writing an AEF and the answers should be absolutely explicit, clear and easily found. The rater needs to know and also the panels need to know. If there are ambiguities and different interpretations - especially between the panels and raters, some employees will be disadvantaged. (For example - if the rater thinks that having checked the box "all WOs were met", he/she does not need to discuss all the WOs and therefore does not do so, the panel may (having a different expectation) construe the silence on one WO as a signal that the WO actually was not met or was done so poorly that there is nothing positive to say about it. That might be the truth or, on the other hand, that might be entirely incorrect. The rater might have said nothing simply because the form is so short and the other WO seemed a livelier topic for discussion.)

- Too little time. The suddenly announced telescoping of the amount of time we had for this process down to two weeks less than we had last year (which I gather was done for reasons of HR convenience) was extremely burdensome. It is from actions of this type that one gleans the understanding that many people in charge of this process are full-time Washington residents and do not have a field perspective.  The amount of time - even under the old schedule - was very compressed. The new schedule made it extraordinarily so. Indeed the new schedule was essentially impossible if one actually took seriously the time allocations given to individual employees to review, respond, write their statements etc. Here are a few field realities - our AEFs have to be DHLed to Washington. DHL here requires three days advance mailing (not including weekends or holidays) to really guarantee delivery. The nice spring months - April and May - are peak times for VIP visits. [...] In addition, out here, we have local holidays and the early part of May is one of the biggest local holiday periods of the entire year. It seems rather clear to me that no one in Washington took these sorts of realities into consideration, because for Washington these sorts of things are not a problem. The result? Several of our employees had less than 24 hours to write their employee statements. And one employee, who'd been locked up day and night in advance meetings, had to write his statement in thirty minutes, with the people hovering, admonishing him that if he did not finish momentarily it would be too late and the Mission 's entire AEF submission would miss the airplane.

- Confusion regarding rated employee's rights. The new procedure of giving the evaluation to the employee before giving it to the AC caused a lot of confusion - in part because the instructions regarding what this process involved were not too clear. We had two employees (one a 4, the other a 3) who were wandering around the Mission complaining bitterly because their rater had declined to include all their proposed changes. They had garnered the impression that the rater (which fortunately wasn't me!) was under some kind of obligation to accept all of the employee's requested revisions. We had to say: Well, no! You are allowed to react, comment, suggest changes - not co-author the document. It would be helpful - not only to raters but also to employees - to state explicitly that the evaluation remains that of the supervisor and that while the rater is required to consider the requested changes, he/she is not required to accept them.

- Lack of clarity about what to do regarding NEPs and new arrivals. We had a devilishly difficult time getting a straight answer regarding what to do about AEFs for a couple of NEPs who arrived at Post about a month before the rating period ended, and had been in language training half the year. We couldn't get AIFs on them from Washington - even though we sent one of our TDYers around to knock on doors at AID/W in person to ask for input. And right up to the 11th hour, we were getting mixed messages about whether we should do an AEF for them at all. In the end, we did.

- In sum, I will look forward to hearing a broader report about overall reaction to the new form and, in particular, how the panels use it. I am hoping for the best but I am concerned that the new form may result in a larger-than-usual number of "mistake" promotions and, over the long term, a reversion to the bad old days when human interactions and treatment of people didn't count for much and it was possible to rise to the highest levels of the agency based solely on technical proficiency and task competence.

11. Feedback on AEF: I think it is a great improvement. One page is quite sufficient to get a sense of a person's performance. So it is half as much agonizing effort for the same result. My impression is that one could have less than 3 work objectives; however, the interpretation at my mission is that one must have 3 objectives. Can you clarify that please? I find the objectives really subjective and fuzzy. One could make up just about anything - it seems rather pointless. The best part of the new system is that the rater must discuss the AEF with the employee before it goes to the board. That is excellent and very necessary. I think they should give a little more space for the employee statement. It is too short to say much.

12. I liked the new version for the long intro section (you can put in a lot of stuff), more liberal space for work objectives (you can make some long and others short), and the condensed space for the text. I found the "skill to work on at next higher level" confusing. This was [HR's] idea, and something [created there]. [They] saw it as a way to motivate an employee, while gently suggesting room for growth. But we were told by HR, part-way through the process, that it isn't an area for improvement. And the guidance, in one place, suggests it could actually be a skill that is superior, not one that needs work. Basically, I found the written guidance somewhat wanting. And there are little errors in the macro (like apparent reference to GS scale, though this is only for FS) that bugged me. [...] I, a couple of years later (and after a blissful AEF-free year at State), had trouble figuring out some things. Maybe people in RRB, who got briefings, knew the answers. And HR gave answers on only one of the questions I raised (that answer came from Lee Roussel, who isn't the answer person). I suggested to David Eckerson that he tell the panels to allow for a lot of variation this year. Some AC members here, and I assume elsewhere, had problems with the concept that they couldn't rewrite or force rewrite. But I think that is a good thing. I also don't see much value added in AC being involved in the work objectives. So that is a good change. And the brevity is wonderful. So many AEFs were redundant before. This way the person has to edit.

Just some quick comments on the AEF process:

* New format is much less painful, and less time consuming, and it provides the same information in less words. I would prefer even more boxes, and even less writing, but this is definitely an improvement.

* I think employees should be given an opportunity to provide feedback on the work of their Mission Directors, and their comments should be taken seriously.

* Supervisors should be required to have a feedback session with their employees after the AEF is completed. Unfortunately, the process does not offer employees any real feedback on what their real strengths and weaknesses are, and how they can improve their performance and which skills they need to improve. In my situation, even when I specifically asked my supervisor and our Mission Director for a feedback session, and followed up, this has not yet happened, and I'm not sure it will happen before I leave post in a couple of months. It's fine to check off boxes, and write about our skills. But, for those employees who are interested in hearing the good, as well as the not so good of their performance (no one does everything well, despite how our AEF narratives end up sounding!), and really are interested in improving their performance, the process is not of much help. I would love to know how my peers and employees rated me, and where they can offer suggestions for improvement; a system should be set in place to allow for that. My AEF was great, and I was outstanding on all the boxes; but I'm sure there are things I can do better, and would like to know about them!!!