Thesis --> Comparing FTF with CMC

Comparing FTF with CMC

Having defined the relevant characteristics of FTF and CMC environments, I was ready to bring this out of the realm of theory and into practice. To this end, I observed and compared the classroom interactions of FTF and CMC courses.

Data collection

Courses

The data for this case study were taken from courses taught at the New York University School of Continuing Education between the fall of 1994 and the summer of 1995.

Some students were attending the school to qualify for a Certificate in Information Management, designed for non-technical managers and professionals. To receive this certificate, students are required to submit proof of satisfactory completion of four courses: an introductory course, two required courses, and an elective.

Due to technical considerations and time constraints, CMC students were required to take all four of the courses for the certificate. The FTF students, on the other hand, could take any of the four courses individually.

In this work, I observed the two required courses: Systems Analysis and Design and Database Management Systems. Both the FTF and CMC versions of the courses taught the same information. The Systems Analysis courses provided an introduction to the role of information technology in organisations and an overview of the systems development life cycle. The Database Management courses were designed to give students a knowledge of the logical structure and physical implementation of database management systems (Accosta, 1994; Accosta, 1995; Vigilante, 1994).

Modes of delivery

The FTF courses were taught during weekday evenings on New York University's campus. The CMC courses were taught by a section of the School of Continuing Education dedicated to distance education, the Virtual College.

The Virtual College began offering courses in 1991 by means of Lotus Notes groupware and modem connections. Notes is a group information manager which provides an electronic environment in which to collect, organise and share information among PCs [8] which are either networked together or connected to a network via modem. Notes permits asynchronous computer conferencing and e-mail. In addition to text, these communications may incorporate data, graphics, audio, and video. Notes also allows the development and sharing of large databases, within which Notes supports hypertext, search, and report functions. Finally, Notes permits the creation of applications to automate these processes (Vigilante, 1994).

Teachers

In the fall of 1994, I observed and tape-recorded FTF courses in Systems Analysis and Database Management. Each course met once a week, for three hours, ten times. The two courses were taught by different lecturers.

In the spring of 1995, I again observed and recorded the same two FTF courses, taught by the same lecturers.

The same two courses were taught via CMC during the summer of 1995, and I observed these by downloading the Lotus Notes databases. The courses were taught by different primary teachers. In addition to the primary teachers, each CMC course had two or three secondary teachers, and the CMC Database Management course had one guest teacher. By primary teacher I mean the main teacher of the course content. A secondary teacher is different from a primary teacher and often has a more specialised role. For example, in the CMC courses there were several teachers who posted messages throughout the courses, but who were not actually teaching the majority of the course content. These secondary teachers were often specialists or technicians who posted specific instructions or answered questions about the software. A guest teacher is a teacher who makes a single appearance to lend his or her expertise to the class. For example, in the CMC Database Management course, the author of one of the required articles posted a response to students' questions.

Fall 1994Spring 1995Summer 1995
Systems Analysis
FTF (Teacher A)XX 
CMC (Teacher B)  X
Database Management
FTF (Teacher C)XX 
CMC (Teacher D)  X

Students

The FTF courses varied between six and 27 students per class with a mean of 17.12 and a median of 18. On average, there were nine males and eight females in each FTF class. For a complete breakdown of attendance by course and class, see Appendix A.

Since the CMC Systems Analysis and Database Management courses were offered as part of a series of four courses [9] for a certificate, the same 19 subjects, 13 male students and six female students, were enrolled in both of the CMC classes. For the purpose of this work, each CMC course was treated as a single class. Even though this is tantamount to ignoring the patterns of a FTF class -- taking attendance at the beginning and summarising the class at the end -- ignoring these is less artificial than arbitrarily dividing a CMC course into sections designed to simulate classes, for CMC has a different social and structural rhythm than FTF interactions (McGrath and Hollingshead, 1990, cited in Sproull and Kiesler, 1991). Although this is confounding, the differences would be barely perceptible in final analyses. For example, the two-to-three minutes of taking attendance in each FTF class would consist of about 1% of the complete FTF course time.

Transcription

Collecting the data

In the FTF courses, interaction between teachers and students was recorded on audiocassette, and was transcribed into Microsoft Word word processing files. For the purpose of transcription, only interaction that took place among all the members of the course was transcribed; interactions between students and the teacher during breaks or after class were not transcribed, nor were discussions which took place among small groups of students [10]. My reasoning was that it was preferable to observe discussion among all course members than attempt to record the in-class interactions of small groups. In each of the courses, one of the sessions was given over to a demonstration of software by vendors. These sessions were not transcribed, as they would reflect the interactions of a sales pitch rather than classroom interaction.

Unfortunately, due to the acoustic properties of the classrooms, some of the words were unclear in the recordings. Although the individual words may not have been decipherable, the number of words spoken and their function in the classroom interaction could often be determined. To permit examination of the classroom talk, these unintelligible words were marked with a tilde (~).

Transcription procedure

To avoid attribution effects, and to ensure objectivity, I ensured that no one who worked on either the transcription, editing, or the coding of the data used in this study had knowledge of all three parts of what was transcribed, which participant said it, and the purposes of this research (Eichler, 1991). I accomplished this in three ways.

First, I ensured that all of the audio tapes were transcribed by transcriptionists blind to the purpose of this research. In addition to transcribing what was said, the transcriptionists also coded the sex and role of the participant who said it.

Second, by means of hiding the text and hiding the coding characters in the FTF transcripts, or, in the case of CMC, using metadatabases [11], I ensured that when I edited the transcripts I could not identify the participant.

Third, I designed the layouts of the database and the hard copy of the transcription so that people coding would not be able to identify the speaker.

Database

Description of the database

In addition to the Word file, I had a FileMaker database which contained all of the text from the transcripts, information about the participant's sex and role, the number of words in each sentence, and metadata about course, class, and term (see Figure 1). If the sentence came from a CMC file, the document's name as well as its date and time would be included. This was unnecessary with data from the FTF transcripts as it could be sorted chronologically by term and class number.

Figure 1: FileMaker Basic Layout

Figure 1: FileMaker Basic Layout

To ensure that the coders did not learn the identities of the speakers from the databases, a layout was created which hid this information (see Figure 2).

Figure 2: FileMaker CHCQS Layout

Figure 2: FileMaker CHCQS [12] Layout

To analyse the transcription from these six courses -- 52,081 sentences -- I taught myself a computer programming language called AppleScript [13] and wrote programs to code and transfer all the transcribed data, by sentence, into databases. I designed these programs to use popular word processing and database software with little modification, permitting other researchers to use these techniques to further their own work.

AppleScript

Most people who use computers are familiar with macros [14]. A macro is a step-saving command, usually implemented through the keyboard, that incorporates several actions or commands. In other words, a macro is a shortcut that combines a memorised series of keystrokes (Schrodt, 1987). For example, a macro in a word processor may be used to insert one's address, often-repeated phrases, or formatting commands.

A scripting language is similar to a macro in that it's designed to incorporate several actions or commands. However, unlike a macro, a script is able to evaluate the state of its existence and proceed based on this data. A scripting language is a simple programming language specifically designed for use with an application. Many applications have incorporated the use of their own scripting language to automate complex tasks [15]. For example, the popular telecommunications program, MicroPhone Pro, permits one to write small scripts to dial a number until a host computer answers (or redial if the line is busy), log on to the computer with username and password (when prompted), summon the mail application, download all new e-mail (if any new mail exists), exit the mail program, log out from the host computer, and hang up. Or, one could write a larger script that would encompass all of the above mentioned functions.

AppleScript differs from all of these in that its use isn't limited to a single application; it can work with several applications at a time, even on different Macintoshes across a network (Schneider, 1994). AppleScript is capable of using the objects and commands within applications by means of Apple events [16].

Apple events are System-level commands or messages sent from one application to another. These messages instruct the receiving application to perform an action or return requested information to the sending application, and form much of the foundation on which AppleScript works (Schneider, 1994). For example, Apple events from the Finder communicate with an application by sending an "open" message when one double-clicks the application's icon.

Apple events provide the infrastructure for applications to communicate with each other. Goodman (1994) uses the analogy that Apple events provide the rails and switches and the applications provide the trains. Apple events can communicate between applications on the same Macintosh, several Macs on a local network, or even Macs connected via modem [17] (DiNucci et al., 1994; Goodman, 1994; Schneider, 1994).

From System 7 onward, Apple events are Apple's overall mechanism for enabling communication between applications [18]. Although important in the way the operating system works, this communication between applications is transparent to users. Normally, if an application is Apple event-aware, the user will only know this if he or she views the available Apple events with the Script Editor's dictionary (see Figure 3).

Figure 3: Dictionary of Apple's Scriptable Text Editor

Figure 3: Dictionary of Apple's Scriptable Text Editor

In August of 1994, when I first considered using AppleScripts to export transcript data by sentence to FileMaker databases (see Development on page 50), Apple event-aware word processors were difficult to find. I briefly considered writing scripts to work with XPress [19]. Upon some reflection, however, I thought it would be better to work with an application which was either free or already commonly used by other people, particularly in educational settings. XPress is a specialised application, designed for professional designers and publishers. Although the computer periodicals proclaimed that the about-to-arrive versions of word processors would be Apple event-aware, the only application then available which fit my criteria was Apple's Scriptable Text Editor.

The Scriptable Text Editor is an extremely basic word processor, about as sophisticated as TeachText or SimpleText, but it is fully Apple event-aware. It is also free. There were, however, two problems with using the Scriptable Text Editor as my text input-output application.

The first problem is that, like TeachText or SimpleText, the Scriptable Text Editor can only handle files that are less than or equal to 32K.

The second problem is that sentence is not listed as an object in its Text suite. As a result, for the script to select a sentence, I originally wrote a script that would start at the beginning of a document and examine each character until it came to a character which I had designated as marking the end of a sentence: exclamation point (!), question mark (?), and full stop or period (.).

Initially I decided to work with these limitations. The 32K limitation could be overcome simply by dividing the original word processing files into segments smaller than 32K before opening them in the Scriptable Text Editor. Although this would require more file maintenance, it was not a prohibitive option.

The lack of sentence in the Text suite presented more of a problem. Although it was non-unique to the Scriptable Text Editor, not being in XPress's Text suite either, examining individual characters to determine when sentences ended proved to be a lengthy process. Merely examining the characters in the sentence "How razorback-jumping frogs can level six piqued gymnasts." would require an examination of 58 characters. Even rounding AppleScript's slothlike performance up to a charitable 2.5-character-per-second processing time [20], that sentence would require over 23 seconds just to be defined as a sentence. Extrapolated to the 2,896,498 characters which constituted the transcripts from the FTF and CMC courses, this would have required over 13 days non-stop merely to define their sentences, let alone have anything done with them.

With this in mind, I decided to find an Apple event-aware word processor that included sentence in its Text suite. Since version 5.1 of Microsoft's Word for Macintosh permits one to select sentences by means of the keystroke command-mouse click, I suspected that the new version would include sentence in its Text suite. A friend who was beta-testing the application confirmed my suspicion and when the new version of Microsoft's Word was released for the Macintosh, I took advantage of an impressive educational discount and bought it.

This is not the forum in which to express my disgust with the Macintosh implementation [21] of Word 6, its hardware requirements [22], or its interface guideline atrocities [23]. However, the reader should be aware that superior word processors with sentence in their Apple event suites may soon -- if not already -- exist.

FTF transcripts

The audio recordings of the FTF courses were given to professional transcriptionists. During the audio tape transcription, each paragraph was prefaced with a coding character to indicate its speaker: "t" for teacher, "m" for male student, and "f" for female student. As an example, an excerpt from the first class of the Fall 1994 Database Management course was coded:

mThe uh, the back of the projector. The overhead projector is on fire.

tOh.

mIt's really smoking.

In addition, in instances where a speaker was interrupted, the sentence was marked with two hyphens as the final character. (For the complete instructions given to the transcriptionists, see Appendix B.)

The transcripts were given to me in WordPerfect for DOS format which I converted to Word 4/5 for Macintosh by means of DataViz's MacLinkPlus [24].

Once I had converted the files, I checked each paragraph to ensure that it was prefaced with a valid coding character. I did this by first replacing the strings which should have been unique to the coding system with red characters, using Word's Replace function. Then, to ensure that I would be ignorant of the content of each speaker's paragraph, I made the text window as narrow as possible. Using my default font of Adobe Palatino at a 12-point size, this permitted me to see a window width of only ten characters -- the coding character as well as the following nine characters of the line. While not as effective as some methods would have been for preserving my ignorance [25], it was adequate. The red letters stood out dramatically from the black text, allowing me to scroll through the file and quickly determine whether valid coding characters had been used.

If an improper coding character had been used, or if a valid coding character was missing, I replaced it with a valid coding character by the following system:

a) If it was obvious that the participant was the teacher I coded it as such (the FTF courses only had one teacher). Giveaway behaviours took the form of calling on students, questioning their answers, and lecturing the course content. Otherwise, I coded it as a student in the manner described in the next paragraph.

b) If the participant was not the teacher, I used my Random Student Selection AppleScript (described in Appendix C) to select "male" or "female" randomly as a function of their class representation [26]. Admittedly, this maintains speaker blindness at the expense of potential validity, as opposed to simply going back to the tape, but for the few sentences this policy affected (approximately three per class) maintaining the integrity of speaker blindness seemed more important. In order to compare FTF and CMC participants fairly, it was important to be blind to who was doing the talking, rather than risk being influenced by my own attributions.

Once I had established that every paragraph began with a valid coding character, I again used Word's Replace function to replace the initial coding character's style from the proportionally-spaced red Palatino to monospaced white Courier. Thus, one could not tell the slim space of an "t" from the wide space of an "m". Having done this, the coding character was hidden, concealing the identity of the speaker [27], but the text remained visible. Now, safely ignorant of the speaker, I was able to edit the transcripts into a consistent format. I began by using the following table and Word's Replace function.

Replace this stringWith this
Two hyphens used to mark an interruption (--)A single em dash [28].
A paragraph mark followed by a coding character ("i" or "m" or "f") and a spaceA paragraph mark followed by the coding character to avoid paragraphs beginning with a space.
Colon, used in instances when transcriptionists wrote "one-to-one" as "1:1"Written out, i.e., "-to-".
A hyphen surrounded on both sides by spaces, used as a dash or an em dash preceded by a spaceThe same thing, but removed the leading space so that the hyphen or em dash would not be counted as a word by itself.
Two tildesTwo tildes with a space between them, so that each tilde would be counted as a word.
Three paragraph marks in a rowTwo paragraph marks in a row. This permits a single blank line, but not two.
A comma followed by an em dashThe em dash only. One is interrupted between words, not punctuation.
A space followed by a paragraph markThe paragraph mark only. This removes excess space at the end of lines.
A space followed by a full stop, comma, or question mark ( .)( ,)( ?)The same less the initial space to ensure that the punctuation will not be counted as a word.
Two consecutive spaces.A single space to remove excess space.
caseCASE, if talking about CASE tools.
main frame, data base, work station, spread sheetmainframe, database, workstation, spreadsheet to ensure a consistent word count.
tabRemoved these by replacing them with nothing.
gooey, sequelGUI (Acronym for Graphical User Interface), SQL (Acronym for Structured Query Language).
PC's, ID's, GUI'sPCs, IDs, GUIs.
schemerschema
SS, dbSocial Security, database
No., vs., v., spec., specs., e.g., i.e., i.d., I.D., etc., ... (three full stops used as an ellipse), .-unspecified digitNumber, vs, v, spec, specs, eg, id, ID, etc, … (ellipse character [29]), -point-number written out. This prevented the punctuation from falsely indicating the end of a sentence.

Once I had completed the relatively automated aspect of preparation, I read through the transcripts and ensured that the punctuation would not cause the AppleScripts to malfunction. I also corrected incidences in which the transcriptionists had inserted homophones due to unfamiliarity with the subject matter. As a result I changed items like "A is 400" to "AS/400," "alda rhythm" to "algorithm," "megadata" to "metadata," and "repressive relationships" to "regressive relationships." [30] When I had completed these tasks, the transcription was ready to be run through the Transcript to Database script (described in Appendix D).

After the FTF and CMC AppleScripts had run, I was left with a very large Word file. It contained all of the exported text, participant sex and role information, and the record numbers for the sentences said within a given paragraph. Using the example from before, the Word file looked like this:

The uh, the back of the projector. The overhead projector is on fire. (Male Student [875-876])

Oh. (Male Teacher [877])

It's really smoking. (Male Student [878])

To hide the participants' identity from coders, yet still show the record numbers to aid access, I made a copy of the file in which the speaker was hidden by replacing "(Male Student ", "(Female Student ", and "(Male Teacher " with nothing, leaving only the record number [31]. For the CMC transcripts I expanded this to include the more extensive coding of "Male Primary Teacher," "Male Secondary Teacher" and "Male Guest Teacher." I was then left with:

The uh, the back of the projector. The overhead projector is on fire. [875-876]

Oh. [877]

It's really smoking. [878]

I printed a hard copy of this file for the coders to view.

CMC transcripts

Although the CMC transcripts were already in computer files, they still had to be put into a form with which AppleScript could work. The participants in the CMC courses communicated via Lotus's Notes. Unfortunately for my purposes, the Macintosh version of Notes we used, 3.30, was not Apple event-aware and all text had to be exported manually from Notes format before I could manipulate it.

To simplify the management of the files, I needed to assign them new file names before exporting the data. To do this, I first exported the list of files from Notes (Figure 4) as text.

Figure 4: List of CMC files in Notes

Figure 4: List of CMC files in Notes

I then imported that text as an Excel spreadsheet and added a column in which I assigned each file a numeric filename (Figure 5).

Figure 5: List of CMC files in Excel

Figure 5: List of CMC files in Excel

I then went through the list, opened the files in Notes (Figure 6) and exported them in Word format with their new numeric names.

Figure 6: CMC file in Notes

Figure 6: CMC file in Notes

Once all of the files had been saved, I removed information about the authors from them to simulate the blind editing conditions I used in the FTF transcripts. To accomplish this, I opened each file in Word and removed the header information, including the author, so that the first line remaining was the date and the second line was the time of posting (see Figure 7). If no time information was included with the file, 12:00 AM was assigned to it. In addition, if the author had signed his or her name at the bottom of the post, it was removed at this time.

If the stripped-down file contained quoted text from another author, it was printed, the quoted text was highlighted, and the printout was set aside. This last step was done so that once the transcripts had been run through the AppleScripts, the quoted text could be marked in the output transcript. It was necessary because in Notes the students indicated that text had been quoted from another source by making it italic or bold. Unfortunately, Word lacks the ability to export styled text via AppleScripts, so everything in the output file is the same style, and the distinction would have been lost.

Figure 7: CMC file in Word

Figure 7: CMC file in Word

When all of the files had been stripped of author identification, I created the CMC metadata database by means of my Metadata Extraction AppleScript. This program opened each file in turn and copied the file name, date and time information to individual records in a FileMaker Pro database (Figure 8). (For a complete description of the Metadata Extraction script, see Appendix E.)

Figure 8: CMC metadata database

Figure 8: CMC metadata database

From the spreadsheet shown in Figure 5, I added information about the file author's sex to the database. I also verified the creation date extracted from the file against the posting date indicated on the spreadsheet, since this occasionally conflicted. In instances of conflict, the posting date was substituted for the creation date to reflect when other participants would have seen it.

In addition, to reflect the varying roles of the various teachers in the CMC courses, the coding character for teacher was expanded to include primary teacher, secondary teacher, and guest teacher.

If necessary, these three categories of teacher could be combined in FileMaker to create a single "teacher" or "not-student" category later. In the FTF database, "Teacher" was recoded as "Primary teacher." In the future, if necessary, FTF transcript coding characters could be revised to include "p" for primary teacher, "s" for secondary teacher, and "g" for guest teachers. More than one sex of any of these, however, would require a more complex coding system.

Also, information about the file's course, term, and whether or not the file included quoted text that had been highlighted and printed out was noted in each record.

Formatting each file in Word was a straightforward affair. Starting with a file set-up like Figure 7, the rest of the document was set up so that line three was blank, and the text -- the original title or subject line of the file -- was on line four. Then, using the command-mouse click keys to check sentences, the text was cleared of punctuation, primarily full stops, that would cause the AppleScript to malfunction. Emoticoms were replaced with "<smile>" for ":)", and "<grin>" for ";)" or ";->".

Once the Word files were prepared for the AppleScripts, I ran them through the Extract Large Sentences script to copy sentences containing more than 254 characters to a text file and replace them with dummy sentences in the original Word files [32]. The transcription was now ready to be run through the CMC to Database script (described in Appendix G).

[Continue]


hillman@cantab.net1997