Methods

SAMPLE:

    Given the time limits of this study (a single semester), I chose a group of musicians to whom I had immediate access. My target sample was a group of core musicians (full-time performers) from the Florida West Coast Symphony (FWCS), the orchestra with which I have been performing for the past two years.  Of the thirty-two core members surveyed, twenty-two responded.  From the pool of responders, twelve volunteered for the interview.  The final count of responses is twenty-two completed surveys and ten completed interviews.  I will not claim external validity for this very small, very specific study.  For generalizing within the orchestra, statistically I can be 95% confident that the margin of error is around 12% given the size of the population and the sample.  

MEASURES:

    Richard Hackman and Erin Lehman, co-authors of the Life and Work in Symphony Orchestras (Allmendinger, Hackman, and Lehman, 1996) study, graciously provided copies of their instruments and analysis (Hackman and Allmendinger, 1990).  From that, I excerpted part of the written survey regarding motivation and satisfaction.  With a few edits and additions aimed at acquiring data about motivations for teaching and the musicians’ satisfaction with career instead of orchestra job, the survey I used was based on their model.  There were questions at the beginning regarding the musician’s career-related activities (performing in other venues, teaching instrumental lessons, and teaching music classes) and the amount of time they had performed as a full-time orchestra musician.  All other responses were on a seven point Likert scale.  (See Appendix B.)

    For the interview, I closely examined the protocol for the Good Work study (Gardner, Csikszentmihalyi, and Damon, 2001).  Many of my questions were alterations of that model, and the others were questions that I constructed in relation to the outcome of the written survey responses.  My intention was to use the interviews to fill information gaps or to illuminate issues that were difficult to interpret from the Likert scale data.  (See Appendix C.)

DESIGN:

    There are many threats to validity.  First and foremost is the relationship of the size of the population and the sample to the scope of the research question.  Though the research question addresses a large generality, I studied only a group within a single orchestra.  Even within the orchestra, the validity is limited and applies specifically to the core musicians.

Within the community of the orchestra, musicians’ contracts are negotiated every three years.  This is a negotiation year, which means that everyone in the organization is thinking and talking about the areas they would like to see improvement.  In addition to this issue, the musicians in the orchestra have been working towards unionization.  The initial vote of intent was in the spring of 2002, but the final vote was on December 4, 2002 after a long dialog within the organization.  My survey was distributed and collected between services on November 21-22, 2002, and I conducted all the interviews after the unionization.  I decided to go ahead with the survey because of time, interest, availability, and the fact that the negotiations or union decision could push the responses in either direction.  There was no certainty that the responses I received would be either more positive or more negative.  Reported job satisfaction could increase based on hopefulness or decrease because more attention was placed on areas of dissatisfaction.  In the interviews, I asked directly about the effects of the negotiation and unionization.  The majority of respondents (7/10) answered that their surveys were not influenced at all, and the others said that the influence was small.  Those who claimed influence mentioned that they felt they were more articulate about the issues than they might be at another time, but that the overall tone of their answers was from the entirety of their experience with orchestras in general.  I will return to this issue in discussing the results.

    Because I have been a member of the orchestra for two years, people treated me as a colleague in the interviews, even though some had joined the organization since my absence.  This could mean they gave me different information or a different kind of response than an unknown interviewer.  I also found a tendency for the respondents to assume that I knew more than I did about the way they were answering my questions, or that I would necessarily share their opinions.  I did ask for clarification on ambiguous points, but I think the possibility of an unusual response to this researcher is worth mentioning.

As this is my first attempt at an independent research study of this kind, I ask the reader to keep in mind a few caveats particular to a novice.  There are certain technical errors a more seasoned researcher might not make.  First, I made an incorrect assumption in the initial concept for the study—that there were many members of the FWCS who did not teach.  Among those musicians in the orchestra with whom I regularly associate on a personal basis, several individuals do not currently teach.  I did not perform a pilot study before the written questionnaire to see if this was the rule for the entire sample.  Perhaps this study could be considered as a pilot for one more suited to ascertain the type of information I initially sought.  Second, I did not have any background in statistics or research prior to this study. Concurrent with this process, I have been studying basic research methods.  My data analysis knowledge is limited, and therefore certainly not exhaustive even for this set of information.  Third, due to the short overall time for the study and the kinds of learning I had to do in order to complete it, I do not feel that I possess all the contextual information regarding this organization to establish causal links.  I postulate potential cause or effect relationships with great caution, hoping to provide directions to seek more information regarding any possibilities.  Fourth, by choosing a sample of my colleagues, I am in a position where neutrality is not my strongest asset.  To the best of my ability, I attempt to present a fair picture of my findings, and hope that they will be interesting and useful to the musicians, management, and board of the FWCS.  I am grateful for the efforts of many members of the organization on my behalf, and for the opportunity to take a year’s leave of absence in order to benefit the organization and myself.

  PROCEDURES:

    I could not disclose the deeper intent to compare teaching to non-teaching musicians to the participants in my study because of the possibility of undue influence.  To protect the authenticity of the results, I gave them a very general view of the goals of this study.  To everyone outside the Harvard community I explained the research as a simple career satisfaction survey.  

    I sent a copy of the survey to the personnel manager of the FWCS to review before it was distributed, in part for feedback and in part to ensure that I had permission to go ahead with the project at an orchestra service.

The surveys were distributed to the members of the core orchestra at a rehearsal and collected within the next two days’ services.  Because I could not be there in person and because I thought I would obtain the greatest possible number of responses, a colleague in the orchestra distributed and gathered the surveys.  The cover letter explained that all responses were confidential and any personally identifying information would not be used.  On the last page of the survey, the respondents were asked to provide their name and telephone number if they would be willing to participate in an interview.  This page was collected and kept separate from the survey responses.

After the surveys were collected, I received a query from the management as to the nature of the information I sought.  There was some fear that my position was somehow anti-management.  Given the circumstances of the unionization debate, I felt it necessary to clarify to the management the true intent of the study, though I am certain that this had no effect on the musicians’ responses to the subsequent interviews.  Where it did affect the outcome was in my respect of their wishes not to pursue the matter further with musicians who might have been disturbed by the initial survey.  I did not attempt to contact non-responders or follow up with any musicians who did not volunteer for the interviews.

The survey information was already in numeric form.  I used the SAS program to analyze the data using means, correlations, scatter plots, t-tests and significance intervals for specific questions and for groupings also used in the Life and Work study (Allmendinger, Hackman, and Lehman, 1996).

After a significant amount of data analysis, I finalized a questionnaire for the interview process.  I scheduled and conducted interviews via telephone using a cassette recorder and taking notes.  Each interview lasted between thirty minutes and two hours.  I replayed and analyzed the interviews to verify my interpretation of the responses.  Due to time constraints, I did not transcribe each interview in full.  In the coding process, I considered issues mentioned at any time during the interview under the code heading to which it applied whether it occurred within that specific question or not. Several people answered multiple areas of inquiry under a single question so not every interview had the same order or number of questions.  The interviewees signed permission and confidentiality agreements and gave recorded verbal consent to the use of their comments.  Their names and identities have been protected.  Any potentially identifying information is used here with full permission.