A large and very important part of Gate One is the client-side JavaScript that runs in the user's browser.  This consists of the following:

Write documentation on embedding Gate One into other applications.  Include details on how to make it hidden until a certain keystroke is entered.

Gate One comes bundled with a number of plugins which can include any number of files in Python, JavaScript, or CSS (yes, you could have a CSS-only plugin!).  These included plugins are below:

Gate One consists of gateone.py and several supporting Python modules and scripts.  The documentation for each can be found below:

#
#       Copyright 2011 Liftoff Software Corporation
#

# Meta
__version__ = '0.9'
__license__ = "AGPLv3 or Proprietary (see LICENSE.txt)"
__version_info__ = (0, 9)
__author__ = 'Dan McDougall <daniel.mcdougall@liftoffsoftware.com>'

# Import stdlib stuff
import sys, re, gzip
from time import sleep
from datetime import datetime
from optparse import OptionParser

# Import our own stuff
from utils import raw

__doc__ = """\
.. _log_viewer:

Log Viewer
==========
Allows the user to play back a given log file like a video (default) or display
it in a syslog-like format.  To view usage information, run it with the --help
switch:

.. ansi-block::
    \x1b[1;31mroot\x1b[0m@host\x1b[1;34m:/opt/gateone $\x1b[0m ./logviewer.py --help
    Usage:  logviewer.py [options] <log file>

    Options:
      --version       show program's version number and exit
      -h, --help      show this help message and exit
      -f, --flat      Display the log line-by-line in a syslog-like format.
      -p, --playback  Play back the log in a video-like fashion. This is the
                    default view.
      --pretty        Preserve font and character renditions when displaying the
                    log in flat view (default).
      --raw           Display control characters and escape sequences when
                    viewing.

Here's an example of how to display a Gate One log (.golog) in a flat, greppable
format:

    \x1b[1;31mroot\x1b[0m@host\x1b[1;34m:/opt/gateone $\x1b[0m ./logviewer.py --flat
    Sep 09 21:07:14 Host/IP or SSH URL [localhost]: modern-host
    Sep 09 21:07:16 Port [22]:
    Sep 09 21:07:16 User: bsmith
    Sep 09 21:07:17 Connecting to: ssh://bsmith@modern-host:22
    Sep 09 21:07:17
    Sep 09 21:07:17 bsmith@modern-host's password:
    Sep 09 21:07:20 Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-11-generic x86_64)
    Sep 09 21:07:20
    Sep 09 21:07:20  * Documentation:  https://help.ubuntu.com/
    Sep 09 21:07:20 Last login: Thu Sep 29 08:51:27 2011 from portarisk
    Sep 09 21:07:20 \x1b[1;34mbsmith\x1b[0m@modern-host\x1b[1;34m:~ $\x1b[0m ls
    Sep 09 21:07:21 why_I_love_gate_one.txt  to_dont_list.txt
    Sep 09 21:07:21 \x1b[1;34mbsmith\x1b[0m@modern-host\x1b[1;34m:~ $\x1b[0m

About Gate One's Log Format
===========================
Gate One's log format (.golog) is a gzip-compressed unicode (UTF-8) text file
consisting of time-based frames separated by the unicode character, U+F0F0F0.
Each frame consists of JavaScript-style timestamp (because it is compact)
followed by a colon and then the text characters of the frame.  A frame ends
when a U+F0F0F0 character is encountered.

Here are two example .golog frames demonstrating the format::

    1317344834868:\\x1b[H\\x1b[2JHost/IP or SSH URL [localhost]: <U+F0F0F>1317344836086:\\r\\nPort [22]: <U+F0F0F>

Gate One logs can be opened, decoded, and parsed in Python fairly easily::

    import gzip
    golog = gzip.open(path_to_golog).read().decode('utf-8')
    for frame in golog.split(u"\U000f0f0f"):
        frame_time = float(frame[:13]) # First 13 chars is the timestamp
        # Timestames can be converted into datetime objects very simply:
        datetime_obj = datetime.fromtimestamp(frame_time/1000)
        frame_text = frame[14:] # This gets you the actual text minus the colon
        # Do something with the datetime_obj and the frame_text

.. note:: U+F0F0F0 is from Private Use Area (PUA) 15 in the Unicode Character Set (UCS). It was chosen at random (mostly =) from PUA-15 because it is highly unlikely to be used in an actual terminal program where it could corrupt a session log.

Class Docstrings
================
"""

# Globals
SEPARATOR = u"\U000f0f0f" # The character used to separate frames in the log

# TODO: Support Fast forward/rewind/pause like Gate One itself.

def playback_log(log_path, file_like, show_esc=False):
    """
    Plays back the log file at *log_path* by way of timely output to *file_like*
    which is expected to be any file-like object with write() and flush()
    methods.

    If *show_esc* is True, escape sequences and control characters will be
    escaped so they can be seen in the output.
    """
    log = gzip.open(log_path).read().decode('utf-8')
    prev_frame_time = None
    for i, frame in enumerate(log.split(SEPARATOR)):
        try:
            frame_time = float(frame[:13]) # First 13 chars is the timestamp
            frame = frame[14:] # Skips the colon
            if i == 0:
                # Write it out immediately
                file_like.write(frame)
                prev_frame_time = frame_time
            else:
            # Wait until the time between the previous frame and now has passed
                wait_time = (frame_time - prev_frame_time)/1000.0
                sleep(wait_time) # frame times are in milliseconds
                if show_esc:
                    frame = raw(frame)
                file_like.flush()
        except ValueError:
            # End of file.  No biggie.
            return

def escape_escape_seq(text, preserve_renditions=True, rstrip=True):
    """
    Escapes escape sequences so they don't muck with the terminal viewing *text*
    Also replaces special characters with unicode symbol equivalents (e.g. so
    you can see what they are without having them do anything to your running
    shell)

    If *preserve_renditions* is True, CSI escape sequences for renditions will
    be preserved as-is (e.g. font color, background, etc).

    If *rstrip* is true, trailing escape sequences and whitespace will be
    removed.
    """
    esc_sequence = re.compile(
        r'\x1b(.*\x1b\\|[ABCDEFGHIJKLMNOQRSTUVWXYZa-z0-9=]|[()# %*+].)')
    csi_sequence = re.compile(r'\x1B\[([?A-Za-z0-9;@:\!]*)([A-Za-z@_])')
    esc_rstrip = re.compile('[ \t]+\x1b.+$')
    #replacement_map = {
        #0: u'␀',
        #7: u'␇',
        #9: u'␉',
        #24: u'␘',
    #}
    out = u""
    esc_buffer = u""
    # If this seems confusing it is because text parsing is a black art! ARRR!
    for char in text:
        if not esc_buffer:
            if char == u'\x1b':
                esc_buffer = char
            #elif ord(char) in replacement_map:
                #out += replacement_map[ord(char)]
            else:
                out += raw(char)
        else:
            esc_buffer += char
            if char == u'\x07' or esc_buffer.endswith(u'\x1b\\'): # Likely title
                esc_buffer = u'' # Nobody wants to see your naked ESC sequence
                continue
            elif esc_buffer.endswith('\x1b\\'):
                esc_buffer = u'' # Ignore
                continue
            # Nobody wants to see plain ESC sequences in the buf...
            match_obj = esc_sequence.match(esc_buffer)
            if match_obj:
                seq_type = match_obj.group(1)
                esc_buffer = u'' # Just when you thought you've ESC'd...
                continue
            # CSI ESC sequences...  These are worth a second look
            match_obj = csi_sequence.match(esc_buffer)
                csi_type = match_obj.group(2)
                if csi_type == 'm' and preserve_renditions: # mmmmmm!
                    out += esc_buffer # Ooh, naked viewing of pretty things!
                esc_buffer = u'' # Make room for more!
                continue
    if rstrip:
        # Remove trailing whitespace + trailing ESC sequences
        return esc_rstrip.sub('', out)
    else: # All these trailers better make for a good movie
        return out

def flatten_log(log_path, preserve_renditions=True, show_esc=False):
    """
    Given a log file at *log_path*, return a list of log lines contained within.

    be preserved as-is (e.g. font color, background, etc).  This is to make the
    output appear as close to how it was originally displayed as possible.
    Besides that, it looks really nice =)

    visible in the output.  Trailing whitespace and escape sequences will not be
    removed.

    NOTE: Converts our standard recording-based log format into something that
    can be used with grep and similar search/filter tools.
    """
    lines = gzip.open(log_path).read().decode('utf-8')
    out = ""
    for frame in lines.split(SEPARATOR):
        try:
            # Convert to datetime object
            frame_time = datetime.fromtimestamp(frame_time/1000)
            if '\n' in frame[14:]: # Skips the colon
                frame_lines = frame[14:].splitlines()
                for i, fl in enumerate(frame_lines):
                    if len(fl):
                        # NOTE: Have to put a rendition reset (\x1b[m) at the
                        # start of each line in case the previous line didn't
                        # reset it on its own.
                        if show_esc:
                            out += "%s %s\n" % ( # Standard Unix log format
                                frame_time.strftime(u'\x1b[m%b %m %H:%M:%S'),
                                raw(fl))
                        else:
                                escape_escape_seq(fl, rstrip=True)
                            )
                    elif i:# Don't need this for the first empty line in a frame
                        out += frame_time.strftime(u'\x1b[m%b %m %H:%M:%S \n')
            elif show_esc:
                if len(out) and out[-1] == '\n':
                    out = u"%s%s\n" % (out[:-1], raw(frame[14:]))
            else:
                escaped_frame = escape_escape_seq(frame[14:], rstrip=False)
                    out = u"%s%s\n" % (out[:-1], escaped_frame)
                elif escaped_frame:
                    # This is pretty much always going to be the first line
                    out += "%s %s\n" % ( # Standard Unix log format
                        frame_time.strftime(u'\x1b[m%b %m %H:%M:%S'),
                        escaped_frame
                    )
        except ValueError as e:
            pass
    return out

if __name__ == "__main__":
    """Parse command line arguments and view the log in the specified format."""
    usage = ('\t%prog [options] <log file>')
    parser = OptionParser(usage=usage, version=__version__)
    parser.disable_interspersed_args()
    parser.add_option("-f", "--flat",
        dest="flat",
        default=False,
        action="store_true",
        help="Display the log line-by-line in a syslog-like format."
    )
    parser.add_option("-p", "--playback",
        dest="playback",
        default=True,
        action="store_false",
        help=("Play back the log in a video-like fashion. This is the default "
              "view.")
    )
    parser.add_option("--pretty",
        dest="pretty",
        help=("Preserve font and character renditions when displaying the log "
              "in flat view (default).")
    )
    parser.add_option("--raw",
        dest="raw",
        help="Display control characters and escape sequences when viewing."
    )
    (options, args) = parser.parse_args()
    if len(args) < 1:
        print("ERROR: You must specify a log file to view.")
        parser.print_help()
        sys.exit(1)
    log_path = args[0]
    if options.flat:
        result = flatten_log(
            log_path, preserve_renditions=options.pretty, show_esc=options.raw)
        print(result)
    else:
        playback_log(log_path, sys.stdout)

#
#

# Meta

About This Module
=================
This crux of this module is the Terminal class which is a pure-Python
implementation of a quintessential Unix-style terminal emulator.  It actually
does its best to emulate an xterm.  This means it supports the majority of the
relevant portions of ECMA-48.  This includes support for emulating varous VT-*
terminal types as well as the "linux" terminal type.

The Terminal class's emulation support is not complete but it should suffice for
most terminal emulation needs.  If additional support for certain escape
sequences or modes are required please feel free to provide a patch or to simply
ask for something to be added.

Note that Terminal was written from scratch in order to be as fast as possible.
Comments have been placed where different implementations/development patterns
have been tried and ultimately failed to provide speed improvements.  Any and
all suggestions or patches to improve speed (or emulation support) are welcome!

Supported Emulation Types
-------------------------
Without any special mode settings or parameters, Terminal should be able to
support most applications under the following terminal types (e.g.
"export TERM=<terminal type>"):

 * xterm (the most important one)
 * ECMA-48/ANSI X3.64
 * Nearly all the VT-* types:  VT-52, VT-100, VT-220, VT-320, VT-420, and VT-520
 * Linux console ("linux")

What Terminal Doesn't Do
------------------------
The Terminal class is meant to emulate the display portion of a given terminal.
It does not translate keystrokes into escape sequences or special control
codes--you'll have to take care of that in your application (or at the
client-side like Gate One).  It does, however, keep track of many
keystroke-specific modes of operation such as Application Cursor Keys and the G0
and G1 charset modes *with* callbacks that can be used to notify your
application when something changes.

Special Considerations
----------------------
Many methods inside Terminal start with an underscore.  This was done to
indicate that such methods shouldn't be called directly (from a program that
imported the module).  If it was thought that a situation might arise where a
method could be used externally by a controlling program, the underscore was
omitted.

Asynchronous Use
----------------
To support asynchronous usage (and make everything faster), Terminal was written
to support extensive callbacks that are called when certain events are
encountered.  Here are the events and their callbacks:

============================    ========================================================================
Callback                        Called when...
Terminal.CALLBACK_SCROLL_UP     The terminal is scrolled up (back).
Terminal.CALLBACK_CHANGED       The screen is changed/updated.
Terminal.CALLBACK_CURSOR_POS    The cursor position changes.
Terminal.CALLBACK_DSR           A Device Status Report (DSR) is requested (via the DSR escape sequence).
Terminal.CALLBACK_TITLE         The terminal title changes (xterm-style)
Terminal.CALLBACK_BELL          The bell character (^G) is encountered.
Terminal.CALLBACK_OPT           The special optional escape sequence is encountered.
Terminal.CALLBACK_MODE          The terminal mode setting changes (e.g. use alternate screen buffer).

Note that Terminal.CALLBACK_DSR is special in that it in most cases it will be called with arguments.  See the code for examples of how and when this happens.

Also, in most cases it is unwise to override Terminal.CALLBACK_MODE since this method is primarily meant for internal use within the Terminal class.

Using Terminal
--------------
Gate One makes extensive use of the Terminal class and its callbacks.  So that's
a great place to look for specific examples (gateone.py and termio.py,
specifically).  Having said that, implementing Terminal is pretty
straightforward::

    >>> import terminal
    >>> term = terminal.Terminal(24, 80)
    >>> term.write("This text will be written to the terminal screen.")
    >>> term.dump()
    [u'This text will be written to the terminal screen.                               ',
    <snip>
    u'                                                                                ']

Here's an example with some basic callbacks:

    >>> def mycallback():
    ...     "This will be called whenever the screen changes."
    ...     print("Screen update! Perfect time to dump the terminal screen.")
    ...     print(term.dump()[0]) # Only need to see the top line for this demo =)
    ...     print("Just dumped the screen.")
    >>> term.callbacks[term.CALLBACK_CHANGED] = mycallback
    >>> term.write("This should result in mycallback() being called")
    Screen update! Perfect time to dump the terminal screen.
    This should result in mycallback() being called
    Just dumped the screen.

.. note:: In testing Gate One it was determined that it is faster to perform the conversion of a terminal screen to HTML on the server side than it is on the client side (via JavaScript anyway).

About The Scrollback Bufffer
----------------------------
The Terminal class implements a scrollback buffer.  Here's how it works:
Whenever a scroll_up() event occurs, the line (or lines) that will be removed
from the top of the screen will be placed into Terminal.scrollback_buf.  Then,
whenever dump_html() is called, the scrollback buffer will be returned along
with the screen output and reset to an empty state.

Why do this?  In the event that a very large write() occurs (e.g. 'ps aux'), it
gives the controlling program the ability to capture what went past the screen
without some fancy tracking logic surrounding Terminal.write().

More information about how this works can be had by looking at the dump_html()
function itself.

.. note:: There's more than one function that empties Terminal.scrollback_buf when called.  You'll just have to have a look around =)

"""

# Imports
import re, time, logging
from collections import defaultdict
from itertools import imap, izip, count
import copy

# Globals
# These are for HTML output:
RENDITION_CLASSES = defaultdict(lambda: None, {
    0: 'reset', # Special: Return everything to defaults
    1: 'bold',
    2: 'dim',
    3: 'italic',
    4: 'underline',
    5: 'blink',
    6: 'fastblink',
    7: 'reverse',
    8: 'hidden',
    9: 'strikethrough',
    10: 'resetfont', # NOTE: The font renditions don't do anything right now
    11: 'font11', # Mostly because I have no idea what they are supposed to look
    12: 'font12', # like.
    13: 'font13',
    14: 'font14',
    15: 'font15',
    16: 'font16',
    17: 'font17',
    18: 'font18',
    19: 'font19',
    20: 'fraktur',
    21: 'boldreset',
    22: 'dimreset',
    23: 'italicreset',
    24: 'underlinereset',
    27: 'reversereset',
    28: 'hiddenreset',
    29: 'strikethroughreset',
    30: 'f0',
    31: 'f1',
    32: 'f2',
    33: 'f3',
    34: 'f4',
    35: 'f5',
    36: 'f6',
    37: 'f7',
    39: 'foregroundreset', # Special: Set FG to default
    40: 'b0',
    41: 'b1',
    42: 'b2',
    43: 'b3',
    44: 'b4',
    45: 'b5',
    46: 'b6',
    47: 'b7',
    49: 'backgroundreset', # Special: Set BG to default
    51: 'frame',
    52: 'encircle',
    53: 'overline',
    60: 'rightline',
    61: 'rightdoubleline',
    62: 'leftline',
    63: 'leftdoubleline'
})

# TODO List:
#
#   * Needs tests!
#   * Figure out how to handle programs like htop that position the cursor without following up with proper rendition reset sequences.

class Terminal(object):
    """
    Terminal controller class.
    """
    ASCII_NUL = 0     # Null
    ASCII_BEL = 7     # Bell (BEL)
    ASCII_BS = 8      # Backspace
    ASCII_HT = 9      # Horizontal Tab
    ASCII_LF = 10     # Line Feed
    ASCII_VT = 11     # Vertical Tab
    ASCII_FF = 12     # Form Feed
    ASCII_CR = 13     # Carriage Return
    ASCII_XON = 17    # Resume Transmission
    ASCII_XOFF = 19   # Stop Transmission or Ignore Characters
    ASCII_CAN = 24    # Cancel Escape Sequence
    ASCII_SUB = 26    # Substitute: Cancel Escape Sequence and replace with ?
    ASCII_ESC = 27    # Escape
    ASCII_CSI = 155   # Control Sequence Introducer (that nothing uses)
    ASCII_HTS = 210   # Horizontal Tab Stop (HTS)

    charsets = {'0': { # Line drawing mode
            95: u' ',
            96: u'◆',
            97: u'▒',
            98: u'\t',
            99: u'\x0c',
            100: u'\r',
            101: u'\n',
            102: u'°',
            103: u'±',
            104: u'\n',
            105: u'\x0b',
            106: u'┘',
            107: u'┐',
            108: u'┌',
            109: u'└',
            110: u'┼',
            111: u'⎺', # All these bars and not a drink!
            112: u'⎻',
            113: u'─',
            114: u'⎼',
            115: u'⎽',
            116: u'├',
            117: u'┤',
            118: u'┴',
            119: u'┬',
            120: u'│',
            121: u'≤',
            122: u'≥',
            123: u'π',
            124: u'≠',
            125: u'£',
            126: u'·' # Centered dot--who comes up with this stuff?!?
        }
    }

    CALLBACK_SCROLL_UP = 1    # Called after a scroll up event (new line)
    CALLBACK_CHANGED = 2      # Called after the screen is updated
    CALLBACK_CURSOR_POS = 3   # Called after the cursor position is updated
    # <waives hand in air> You are not concerned with the number 4
    CALLBACK_DSR = 5          # Called when a DSR requires a response
    # NOTE: CALLBACK_DSR must accept 'response' as either the first argument or
    #       as a keyword argument.
    CALLBACK_TITLE = 6        # Called when the terminal sets the window title
    CALLBACK_BELL = 7         # Called after ASCII_BEL is encountered.
    CALLBACK_OPT = 8   # Called when we encounter the optional ESC sequence
    # NOTE: CALLBACK_OPT must accept 'chars' as either the first argument or as
    #       a keyword argument.
    CALLBACK_MODE = 9  # Called when the terminal mode changes (e.g. DECCKM)

    RE_CSI_ESC_SEQ = re.compile(r'\x1B\[([?A-Za-z0-9;@:\!]*)([A-Za-z@_])')
    RE_ESC_SEQ = re.compile(r'\x1b(.*\x1b\\|[ABCDEFGHIJKLMNOQRSTUVWXYZa-z0-9=]|[()# %*+].)')
    RE_TITLE_SEQ = re.compile(r'\x1b\][0-2]\;(.*)(\x07|\x1b\\)')
    # The below regex is used to match our optional (non-standard) handler
    RE_OPT_SEQ = re.compile(r'\x1b\]_\;(.*)(\x07|\x1b\\)')
    RE_NUMBERS = re.compile('\d*') # Matches any number

    def __init__(self, rows=24, cols=80):
        self.cols = cols
        self.rows = rows
        self.scrollback_buf = []
        self.scrollback_renditions = []
        self.title = "Gate One"
        self.ignore = False
        self.local_echo = True
        self.esc_buffer = '' # For holding escape sequences as they're typed.
        self.prev_esc_buffer = '' # Special: So we can differentiate between
                                  # certain circumstances.
        self.show_cursor = True
        self.rendition_set = False
        self.init_screen()
        self.init_renditions()
        self.G0_charset = 'B'
        self.G1_charset = 'B'
        # Set the default window margins
        self.top_margin, self.bottom_margin = 0, self.rows - 1

        self.specials = {
            self.ASCII_NUL: self.__ignore,
            self.ASCII_BEL: self._bell,
            self.ASCII_BS: self._backspace,
            self.ASCII_HT: self._horizontal_tab,
            self.ASCII_LF: self._linefeed,
            self.ASCII_VT: self._linefeed,
            self.ASCII_FF: self._linefeed,
            self.ASCII_CR: self._carriage_return,
            self.ASCII_XON: self._xon,
            self.ASCII_CAN: self._cancel_esc_sequence,
            self.ASCII_XOFF: self._xoff,
            self.ASCII_ESC: self._sub_esc_sequence,
            self.ASCII_ESC: self._escape,
            self.ASCII_CSI: self._set_rendition,
        }
        # TODO: Finish these:
        self.esc_handlers = {
            # TODO: Make a different set of these for each respective emulation mode (VT-52, VT-100, VT-200, etc etc)
            '#': self._set_line_params, # Varies
            '\\': self._string_terminator, # ST
            'c': self.clear_screen, # Reset terminal
            'D': self.__ignore, # Move/scroll window up one line    IND
            'M': self._reverse_linefeed, # Move/scroll window down one line RI
            'E': self._next_line, # Move to next line NEL
            'F': self.__ignore, # Enter Graphics Mode
            'G': self._next_line, # Exit Graphics Mode
            '6': self._dsr_get_cursor_position, # Get cursor position   DSR
            '7': self.save_cursor_position, # Save cursor position and attributes   DECSC
            '8': self.restore_cursor_position, # Restore cursor position and attributes   DECSC
            'H': self._set_tabstop, # Set a tab at the current column   HTS
            'I': self._reverse_linefeed,
            '(': self.set_G0_charset, # Designate G0 Character Set
            ')': self.set_G1_charset, # Designate G1 Character Set
            'N': self.__ignore, # Set single shift 2    SS2
            'O': self.__ignore, # Set single shift 3    SS3
            '5': self._device_status_report, # Request: Device status report DSR
            '0': self.__ignore, # Response: terminal is OK  DSR
            'P': self._dcs_handler, # Device Control String  DCS
            '=': self.__ignore, # Application Keypad  DECPAM
            '>': self.__ignore, # Exit alternate keypad mode
            '<': self.__ignore, # Exit VT-52 mode
        }
        self.csi_handlers = {
            'A': self.cursor_up,
            'B': self.cursor_down,
            'C': self.cursor_right,
            'D': self.cursor_left,
            'E': self.cursor_next_line,
            'F': self.cursor_previous_line,
            'G': self.cursor_horizontal_absolute,
            'H': self.cursor_position,
            'L': self.insert_line,
            #'M': self.delete_line, # TODO
            #'b': self.repeat_last_char, # TODO
            'c': self._csi_device_status_report, # Device status report (DSR)
            'g': self.__ignore, # TODO: Tab clear
            'h': self._set_expanded_mode,
            'l': self._reset_expanded_mode,
            'f': self.cursor_position,
            'd': self.cursor_position_vertical, # Vertical Line Position Absolute (VPA)
            #'e': self.cursor_position_vertical_relative, # VPR TODO
            'J': self.clear_screen_from_cursor,
            'K': self.clear_line_from_cursor,
            'S': self.scroll_up,
            'T': self.scroll_down,
            's': self.save_cursor_position,
            'u': self.restore_cursor_position,
            'm': self._set_rendition,
            'n': self.__ignore, # <ESC>[6n is the only one I know of (request cursor position)
            #'m': self.__ignore, # For testing how much CPU we save when not processing CSI
            'p': self.terminal_reset, # TODO: "!p" is "Soft terminal reset".  Also, "Set conformance level" (VT100, VT200, or VT300)
            'r': self._set_top_bottom, # DECSTBM (used by many apps)
            'q': self.set_led_state, # Seems a bit silly but you never know
            'P': self.delete_characters, # DCH Deletes the specified number of chars
            'X': self._erase_characters, # ECH Same as DCH but also deletes renditions
            'Z': self.insert_characters, # Inserts the specified number of chars
            '@': self.insert_characters, # Inserts the specified number of chars
            #'`': self._char_position_row, # Position cursor (row only)
            #'t': self.window_manipulation, # TODO
            #'z': self.locator, # TODO: DECELR "Enable locator reporting"
        }
        self.expanded_modes = {
            # Expanded modes take a True/False argument for set/reset
            '1': self.application_mode,
            '2': self.__ignore, # DECANM and set VT100 mode
            '3': self.__ignore, # 132 Column Mode (DECCOLM)
            '4': self.__ignore, # Smooth (Slow) Scroll (DECSCLM)
            '5': self.__ignore, # Reverse video (might support in future)
            '6': self.__ignore, # Origin Mode (DECOM)
            '7': self.__ignore, # Wraparound Mode (DECAWM)
            '8': self.__ignore, # Auto-repeat Keys (DECARM)
            '9': self.__ignore, # Send Mouse X & Y on button press (maybe)
            '12': self.send_receive_mode, # SRM
            '18': self.__ignore, # Print form feed (DECPFF)
            '19': self.__ignore, # Set print extent to full screen (DECPEX)
            '25': self.show_hide_cursor,
            '38': self.__ignore, # Enter Tektronix Mode (DECTEK)
            '41': self.__ignore, # more(1) fix (whatever that is)
            '42': self.__ignore, # Enable Nation Replacement Character sets (DECNRCM)
            '44': self.__ignore, # Turn On Margin Bell
            '45': self.__ignore, # Reverse-wraparound Mode
            '46': self.__ignore, # Start Logging (Hmmm)
            '47': self.alternate_screen_buffer, # Use Alternate Screen Buffer
            '66': self.__ignore, # Application keypad (DECNKM)
            '67': self.__ignore, # Backarrow key sends delete (DECBKM)
            '1000': self.__ignore, # Send Mouse X/Y on button press and release
            '1001': self.__ignore, # Use Hilite Mouse Tracking
            '1002': self.__ignore, # Use Cell Motion Mouse Tracking
            '1003': self.__ignore, # Use All Motion Mouse Tracking
            '1010': self.__ignore, # Scroll to bottom on tty output
            '1011': self.__ignore, # Scroll to bottom on key press
            '1035': self.__ignore, # Enable special modifiers for Alt and NumLock keys
            '1036': self.__ignore, # Send ESC when Meta modifies a key
            '1037': self.__ignore, # Send DEL from the editing-keypad Delete key
            '1047': self.__ignore, # Use Alternate Screen Buffer
            '1048': self.__ignore, # Save cursor as in DECSC
            '1049': self.alternate_screen_buffer_cursor, # Save cursor as in DECSC and use Alternate Screen Buffer, clearing it first
            '1051': self.__ignore, # Set Sun function-key mode
            '1052': self.__ignore, # Set HP function-key mode
            '1060': self.__ignore, # Set legacy keyboard emulation (X11R6)
            '1061': self.__ignore, # Set Sun/PC keyboard emulation of VT220 keyboard
        }
        self.callbacks = {
            self.CALLBACK_SCROLL_UP: None,
            self.CALLBACK_CHANGED: None,
            self.CALLBACK_CURSOR_POS: None,
            self.CALLBACK_DSR: None,
            self.CALLBACK_TITLE: None,
            self.CALLBACK_BELL: None,
            self.CALLBACK_OPT: None,
            self.CALLBACK_MODE: None
        }
        self.leds = {
            1: False,
            2: False,
            3: False,
            4: False
        }
        # These are for saving self.screen and self.renditions so we can support
        # an "alternate buffer"
        self.alt_screen = None
        self.alt_renditions = None
        self.alt_cursorX = 0
        self.alt_cursorY = 0
        self.saved_cursorX = 0
        self.saved_cursorY = 0
        self.saved_rendition = [None]
        self.application_keys = False
        self.__linecounter = count()

    def init_screen(self):
        """
        Fills self.screen with empty lines of (unicode) spaces using self.cols
        and self.rows for the dimensions.

        NOTE: Just because each line starts out with a uniform length does not
        mean it will stay that way.  Processing of escape sequences is handled
        when an output function is called.
        """
        self.screen = [
            [u' ' for a in xrange(self.cols)] for b in xrange(self.rows)
        ]
        # Tabstops
        tabs, remainder = divmod(self.cols, 8) # Default is every 8 chars
        self.tabstops = [(a*8)-1 for a in xrange(tabs)]
        self.tabstops[0] = 0 # Fix the first tabstop (which will be -1)
        # Base cursor position
        self.cursorX = 0
        self.cursorY = 0

    def init_renditions(self):
        """
        Fills self.renditions with lists of None using self.cols and self.rows
        for the dimenions.
        """
        self.renditions = [
            [None for a in xrange(self.cols)] for b in xrange(self.rows)
        ]

    def init_scrollback(self):
        """
        Empties out the scrollback buffers
        """

    def __reset_linecounter(self):
        """
        Resets self.linecounter
        """

    def terminal_reset(self, *args, **kwargs):
        """
        Resets the terminal back to an empty screen with all defaults.
        """
            1: False,
            2: False,
            3: False,
            4: False
        }
        self.esc_buffer = ''
        self.prev_esc_buffer = ''
        self.init_scrollback()

    def __ignore(self, *args, **kwargs):
        """Do nothing"""
        pass

    def resize(self, rows, cols):
        """
        Resizes the terminal window, adding or removing rows or columns as
        needed.
        """
        if rows < self.rows: # Remove rows from the top
            for i in xrange(self.rows - rows):
                self.screen.pop(0)
                self.renditions.pop(0)
        elif rows > self.rows: # Add rows at the bottom
            for i in xrange(rows - self.rows):
                line = [u' ' for a in xrange(cols)]
                renditions = [None for a in xrange(self.cols)]
                self.screen.append(line)
                self.renditions.append(renditions)
        self.top_margin = 0
        self.bottom_margin = self.rows - 1

        # Fix the cursor location:
        if self.cursorY >= self.rows:
            self.cursorY = self.rows - 1

        # TODO: Test these to make sure they're sane in all situations
        if cols < self.cols: # Remove cols to the right
            for i in xrange(self.rows):
                self.screen[i] = self.screen[i][:cols - self.cols]
                self.renditions[i] = self.renditions[i][:cols - self.cols]
        elif cols > self.cols: # Add cols to the right
                for j in xrange(cols - self.cols):
                    self.screen[i].append(u' ')
                    self.renditions[i].append(None)

        if self.cursorX >= self.cols:
            self.cursorX = self.cols - 1

    def _set_top_bottom(self, settings):
        """
        DECSTBM - Sets self.top_margin and self.bottom_margin using the provided
        settings in the form of '<top_margin>;<bottom_margin>'
        """
        # NOTE: Used by screen and vi so this needs to work and work well!
        if len(settings):
            top, bottom = settings.split(';')
            self.top_margin = max(0, int(top) - 1) # These are 0-based like self.cursor[XY]
            if bottom:
                self.bottom_margin = min(self.rows - 1, int(bottom) - 1)
        else:
            # Reset to defaults (full screen margins)
            self.top_margin, self.bottom_margin = 0, self.rows - 1

    def get_cursor_position(self):
        """
        Returns the current cursor positition as a tuple, (row, col)
        """
        return (self.cursorY, self.cursorX)

    def set_title(self, title):
        """
        Sets self.title to *title* and executes
        self.callbacks[self.CALLBACK_TITLE]()
        """
        self.title = title
        try:
            self.callbacks[self.CALLBACK_TITLE]()
        except TypeError as e:
            print("Got TypeError on CALLBACK_TITLE...")
            print(repr(self.callbacks[self.CALLBACK_TITLE]))
            print(e)

# TODO: put some logic in these save/restore functions to walk the current
# rendition line to come up with a logical rendition for that exact spot.
    def save_cursor_position(self):
        """
        Saves the cursor position and current rendition settings to
        self.saved_cursorX, self.saved_cursorY, and self.saved_rendition
        """
        self.saved_cursorX = self.cursorX
        self.saved_cursorY = self.cursorY
        self.saved_rendition = self.renditions[self.cursorY][self.cursorX]

    def restore_cursor_position(self):
        """
        Restores the cursor position and rendition settings from
        self.saved_cursorX, self.saved_cursorY, and self.saved_rendition (if
        they're set).
        """
        if self.saved_cursorX and self.saved_cursorY:
            self.cursorX = self.saved_cursorX
            self.cursorY = self.saved_cursorY
            self.renditions[self.cursorY][self.cursorX] = self.saved_rendition

    def _dsr_get_cursor_position(self):
        """
        Returns the current cursor positition as a DSR response in the form of:
        '\x1b<self.cursorY>;<self.cursorX>R'.  Also executes CALLBACK_DSR with
        the same output as the first argument.  Example:

            self.callbacks[self.CALLBACK_DSR]('\x1b20;123R')
        """
        esc_cursor_pos = '\x1b%s;%sR' % (self.cursorY, self.cursorX)
        try:
            self.callbacks[self.CALLBACK_DSR](esc_cursor_pos)
        except TypeError:
            pass
        return esc_cursor_pos

    def _dcs_handler(self, string=None):
        """
        Handles Device Control String sequences.  Still haven't figured out if
        these really need to be implemented.
        """
        print("TODO: Handle this DCS: %s" % string)

    def _set_line_params(self, param):
        """
        This function handles the control sequences that set double and single
        line heights and widths.  NOTE: Not actually implemented yet!
        """
        print("TODO: Handle this line height setting: %s" % param)

    def set_G0_charset(self, char):
        """
        Sets the terminal's G0 (default) charset to the type specified by *char*

        Here's the possibilities:
            0    DEC Special Character and Line Drawing Set
            A    United Kingdom (UK)
            B    United States (USASCII)
            4    Dutch
            C    Finnish
            5    Finnish
            R    French
            Q    French Canadian
            K    German
            Y    Italian
            E    Norwegian/Danish
            6    Norwegian/Danish
            Z    Spanish
            H    Swedish
            7    Swedish
            =    Swiss

        NOTE: Doesn't actually do anything other than set the variable.
        """
        self.G0_charset = char

    def set_G1_charset(self, char):
        """
        Sets the terminal's G1 (alt) charset to the type specified by *char*

            4    Dutch
            C    Finnish
            5    Finnish
            R    French
            K    German
            Y    Italian
            Z    Spanish
            H    Swedish
            7    Swedish
            =    Swiss

        """
        self.G1_charset = char

    def write(self, chars):
        """
        Write *chars* to the terminal at the current cursor position advancing
        the cursor as it does so.  If *chars* is not unicode, it will be
        converted to unicode before being stored in self.screen.
        """
        # Speedups (don't want dots in loops if they can be avoided)
        specials = self.specials
        esc_handlers = self.esc_handlers
        csi_handlers = self.csi_handlers
        RE_ESC_SEQ = self.RE_ESC_SEQ
        RE_CSI_ESC_SEQ = self.RE_CSI_ESC_SEQ
        cursor_right = self.cursor_right
        changed = False
        for char in chars:
            charnum = ord(char)
            if charnum in specials:
                specials[charnum]()
            elif not self.ignore:
                # Now handle the regular characters and escape sequences
                if self.esc_buffer: # We've got an escape sequence going on...
                    try:
                        self.esc_buffer += char
                        # First try to handle non-CSI ESC sequences (the basics)
                        match_obj = RE_ESC_SEQ.match(self.esc_buffer)
                        if match_obj:
                            seq_type = match_obj.group(1) # '\x1bA' -> 'A'
                            # Call the matching ESC handler
                            if len(seq_type) == 1: # Single-character sequnces
                                esc_handlers[seq_type]()
                            else: # Multi-character stuff like '\x1b)B'
                                esc_handlers[seq_type[0]](seq_type[1:])
                            self.prev_esc_buffer = self.esc_buffer
                            self.esc_buffer = '' # All done with this one
                            continue
                        # Next try to handle CSI ESC sequences
                        match_obj = RE_CSI_ESC_SEQ.match(self.esc_buffer)
                            csi_values = match_obj.group(1) # e.g. '0;1;37'
                            csi_type = match_obj.group(2) # e.g. 'm'
                            if csi_type == 'm':
                                self.rendition_set = True
                            # Call the matching CSI handler
                            csi_handlers[csi_type](csi_values)
                            self.esc_buffer = ''
                            continue
                    except KeyError:
                        # No handler for this, try some alternatives
                        if self.esc_buffer.endswith('\x1b\\'):
                            self._osc_handler()
                        else:
                            print(
                                "Warning: No ESC sequence handler for %s"
                                % `self.esc_buffer`
                            )
                    continue # We're done here
                # TODO: Figure out a way to write characters past the edge of the screen so that users can copy & paste without having newlines in the middle of everything.
                if self.local_echo:
                    changed = True
                    if self.cursorX >= self.cols:
                        # Start a newline but NOTE: Not really the best way to
                        # handle this because it means copying and pasting lines
                        # will end up broken into pieces of size=self.cols
                        self._newline()
                        self.cursorX = 0
                    if not self.rendition_set:
                        # If no rendition was just set on this position empty it
                        self.renditions[self.cursorY][self.cursorX] = None
                    if self.G0_charset == '0':
                        self.screen[self.cursorY][self.cursorX] = self.charsets[
                            '0'][charnum]
                    else:
                        self.screen[self.cursorY][self.cursorX] = unicode(char)
                    self.prev_esc_buffer = ''
                    cursor_right()
                self.rendition_set = False
        if changed:
            # Execute our callbacks
            try:
                self.callbacks[self.CALLBACK_CHANGED]()
            except TypeError:
                pass
            try:
                self.callbacks[self.CALLBACK_CURSOR_POS]()
                pass

    def flush(self):
        """
        Only here to make Terminal compatible with programs that want to use
        file-like methods.
        """
        pass

    def scroll_up(self, n=1):
        """
        Scrolls up the terminal screen by *n* lines (default: 1). The callbacks
        CALLBACK_CHANGED and CALLBACK_SCROLL_UP are called after scrolling the
        screen.

        NOTE: This will only scroll up the region within self.top_margin and
        self.bottom_margin (if set).
        """
        for x in xrange(int(n)):
            line = self.screen.pop(self.top_margin) # Remove the top line
            self.scrollback_buf.append(line) # Add it to the scrollback buffer
            empty_line = [u' ' for a in xrange(self.cols)] # Line full of spaces
            # Add it to the bottom of the window:
            self.screen.insert(self.bottom_margin, empty_line)
            # Remove top line's style information
            style = self.renditions.pop(self.top_margin)
            self.scrollback_renditions.insert(self.bottom_margin, style)
            # Insert a new empty rendition as well:
            self.renditions.insert(
                self.bottom_margin, [None for a in xrange(self.cols)])
        # Execute our callback indicating lines have been updated
        try:
            self.callbacks[self.CALLBACK_CHANGED]()
            pass

        # Execute our callback to scroll up the screen
        try:
            self.callbacks[self.CALLBACK_SCROLL_UP]()
            pass

    def scroll_down(self, n=1):
        """
        Scrolls down the terminal screen by *n* lines (default: 1). The
        callbacks CALLBACK_CHANGED and CALLBACK_SCROLL_DOWN are called after
        scrolling the screen.
        """
            line = self.screen.pop(self.bottom_margin) # Remove the bottom line
            self.screen.insert(self.top_margin, empty_line) # Add it to the top
            # Remove bottom line's style information:
            style = self.renditions.pop(self.bottom_margin)
            # Insert a new empty one:
                self.top_margin, [None for a in xrange(self.cols)])
        try:
            pass

        try:
            pass

    def insert_line(self, n):
        """
        Inserts a line at the current cursor position.
        """
        line = self.screen.pop(self.bottom_margin) # Remove the bottom line
        # Remove bottom line's style information as well:
        style = self.renditions.pop(self.bottom_margin)
        empty_line = [u' ' for a in xrange(self.cols)] # Line full of spaces
        self.screen.insert(self.cursorY, empty_line) # Insert at cursor
        # Insert a new empty rendition as well:
        self.renditions.insert(self.cursorY, [None for a in xrange(self.cols)])

    def _backspace(self):
        """Execute a backspace (\x08)"""
        self.cursor_left(1)

    def _horizontal_tab(self):
        """Execute horizontal tab (\x09)"""
        next_tabstop = self.cols -1
        for tabstop in self.tabstops:
            if tabstop > self.cursorX:
                next_tabstop = tabstop
                break
        self.cursorX = next_tabstop

    def _set_tabstop(self):
        """Sets a tabstop at the current position of self.cursorX."""
        if self.cursorX not in self.tabstops:
            for tabstop in self.tabstops:
                if self.cursorX > tabstop:
                    self.tabstops.append(self.cursorX)
                    self.tabstops.sort() # Put them in order :)
                    break

    def _linefeed(self):
        """Execute line feed"""
        self._newline()

    def _next_line(self):
        """Moves cursor down one line"""
        if self.cursorY < self.rows -1:
            self.cursorY += 1

    def _reverse_linefeed(self):
        self.cursorY -= 1
        if self.cursorY < self.top_margin:
            self.scroll_down()
            self.cursorY = self.top_margin

    def _newline(self):
        """
        Adds a new line to self.screen and sets self.cursorX to 0.
        """
        self.cursorY += 1
        if self.cursorY > self.bottom_margin:
            self.scroll_up()
            self.cursorY = self.bottom_margin
            self.clear_line()

    def _carriage_return(self):
        """
        Execute carriage return (set self.cursorX to 0)
        """

    def _xon(self):
        """
        Handle XON character (stop ignoring)
        """

    def _xoff(self):
        """
        Handle XOFF character (start ignoring)
        """
        self.ignore = True

    def _cancel_esc_sequence(self):
        """Cancels any escape sequence currently in progress."""

    def _sub_esc_sequence(self):
        """
        Cancels any escape sequence currently in progress and substitutes it
        with single question mark (?).
        """
        self.write('?')

    def _escape(self):
        """
        Handle escape character as well as escape sequences.
        """
        buf = self.esc_buffer
        if buf.startswith('\x1bP') or buf.startswith('\x1b]'):
            # CSRs and OSCs are special
            self.esc_buffer += '\x1b'
        else:
            # Get rid of whatever's there since we obviously didn't know what to
            # do with it
            self.esc_buffer = '\x1b'

    def _string_terminator(self):
        """
        Handle the string terminator.
        """
        # TODO: This.
        # NOTE: Might this just call _cancel_esc_sequence?  I need to double-check.
        pass

    def _osc_handler(self):
        """
        Handles Operating System Command (OSC) escape sequences which need
        special care since they are of indeterminiate length and end with either
        a bell (\x07) or a sequence terminator (\x9c aka ST).  This will usually
        called from self._bell() to set the title of the terminal (just like an
        xterm) but it is also possible to be called directly whenever an ST is
        encountered.
        """
        # Try the title sequence first
        match_obj = self.RE_TITLE_SEQ.match(self.esc_buffer)
        if match_obj:
            title = match_obj.group(1)
            self.set_title(title) # Sets self.title
            self.esc_buffer = ''
            return
        # Next try our special optional handler sequence
        match_obj = self.RE_OPT_SEQ.match(self.esc_buffer)
            text = match_obj.group(1)
            self.__opt_handler(text)
            return

    def _bell(self):
        """
        Handle bell character and execute self.CALLBACKS[CALLBACK_BELL]() if we
        are not in the middle of an escape sequence.  If we *are* in the middle
        of an escape sequence, call self._osc_handler() since we can be nearly
        certain that we're simply terminating an OSC sequence.
        """
        # NOTE: A little explanation is in order: The bell character (\x07) by
        #       itself should play a bell (pretty straighforward).  However, if
        #       the bell character is at the tail end of a particular escape
        #       sequence (string starting with \x1b]0;) this indicates an xterm
        #       title (everything between \x1b]0;...\x07).
        if not self.esc_buffer: # We're not in the middle of an esc sequence
            try:
                self.callbacks[self.CALLBACK_BELL]()
                pass
        else: # We're (likely) setting a title
            self.esc_buffer += '\x07' # Add the bell char so we don't lose it
            self._osc_handler()

    def _device_status_report(self):
        """
        Returns '\x1b[0n' (terminal OK) and executes
        self.callbacks[self.CALLBACK_DSR]("\x1b[0n").
        """
        response = "\x1b[0n"
        try:
            self.callbacks[self.CALLBACK_DSR](response)
            pass
        return response

    def _csi_device_status_report(self, request):
        """
        Returns '\x1b[1;2c' (Meaning: I'm a vt220 terminal, version 1.0) and
        executes self.callbacks[self.CALLBACK_DSR]("\x1b[1;2c").
        """
        response = "\x1b[1;2c"
        try:
            pass

    def _set_expanded_mode(self, setting):
        """
        Accepts "standard mode" settings.  Typically '\x1b[?25h' to hide cursor.

        Notes on modes::
            '?1h' - Application Cursor Keys
            '?12h' - Local echo (SRM or Send Receive Mode)
            '?25h' - Hide cursor
            '?1049h' - Save cursor and screen
        """
        # TODO: Add support for the following:
        # * 3: 132 column mode (might be "or greater")
        # * 4: Smooth scroll (for animations and also makes things less choppy)
        # * 5: Reverse video (should be easy: just need some extra CSS)
        # * 6: Origin mode
        # * 7: Wraparound mode
        setting = setting[1:] # Don't need the ?
        settings = setting.split(';')
        for setting in settings:
            try:
                self.expanded_modes[setting](True)
            except (KeyError, TypeError):
                pass # Unsupported expanded mode
        try:
            self.callbacks[self.CALLBACK_MODE](setting, True)
            pass

    def _reset_expanded_mode(self, setting):
        """
        Accepts "standard mode" settings.  Typically '\x1b[?25l' to show cursor.
        """
            try:
                self.expanded_modes[setting](False)
        try:
            self.callbacks[self.CALLBACK_MODE](setting, False)
            pass

    def application_mode(self, boolean):
        """self.application_keys = *boolean*"""
        self.application_keys = boolean

    def alternate_screen_buffer(self, alt):
        """
        If *alt* is True, copy the current screen and renditions to
        self.alt_screen and self.alt_renditions then re-init self.screen and
        self.renditions.
        If *alt* is False, restore the saved screen buffer and renditions then
        nullify self.alt_screen and self.alt_renditions.
        """
        if alt:
            # Save the existing screen and renditions
            self.alt_screen = copy.copy(self.screen)
            self.alt_renditions = copy.copy(self.renditions)
            # Make a fresh one
            self.clear_screen()
        else:
            # Restore the screen
            if self.alt_screen and self.alt_renditions:
                self.screen = self.alt_screen
                self.renditions = self.alt_renditions
            # Empty out the alternate buffer (to save memory)
            self.alt_screen = None
            self.alt_renditions = None

    def alternate_screen_buffer_cursor(self, alt):
        """
        Same as self.alternate_screen_buffer but saves/restores the cursor
        location.
        """
        if alt:
            self.alt_cursorX = self.cursorX
            self.alt_cursorY = self.cursorY
        else:
            self.cursorX = self.alt_cursorX
            self.cursorY = self.alt_cursorY
        self.alternate_screen_buffer(alt)

    def show_hide_cursor(self, boolean):
        """self.show_cursor = boolean"""
        self.show_cursor = boolean

    def send_receive_mode(self, onoff):
        """
        Turns on or off local echo dependong on the value of *onoff*

        self.local_echo = *onoff*
        """
        if onoff:
            self.local_echo = False
        else:
            self.local_echo = True

    def insert_characters(self, n=1):
        """Inserts the specified number of characters at the cursor position"""
        n = int(n)
        for i in xrange(n):
            self.screen[self.cursorY].pop() # Take one down, pass it around
            self.screen[self.cursorY].insert(self.cursorX, u' ')

    def delete_characters(self, n=1):
        """
        DCH - Deletes (to the left) the specified number of characters at the
        cursor position.  As characters are deleted, the remaining characters
        between the cursor and right margin move to the left. Character
        attributes (renditions) move with the characters.  The terminal adds
        blank spaces with no visual character attributes at the right margin.
        DCH has no effect outside the scrolling margins.

        NOTE: Deletes renditions too.
        """
        if not n: # e.g. n == ''
            n = 1
        else:
            n = int(n)
            try:
                self.screen[self.cursorY].pop(self.cursorX)
                self.screen[self.cursorY].append(u' ')
                self.renditions[self.cursorY].pop(self.cursorX)
                self.renditions[self.cursorY].append(None)
            except IndexError:
                # At edge of screen, ignore
                pass

    def _erase_characters(self, n=1):
        """
        Erases (to the right) the specified number of characters at the cursor
        position.  NOTE: Deletes renditions too.
        """
            n = 1
        else:
            n = int(n)
        distance = self.cols - self.cursorX
        n = min(n, distance)
            self.screen[self.cursorY][self.cursorX+i] = u' '
            self.renditions[self.cursorY][self.cursorX+i] = None

    def cursor_left(self, n=1):
        """ESCnD CUB (Cursor Back)"""
        n = int(n)
        self.cursorX = max(0, self.cursorX - n)
        try:
            self.callbacks[self.CALLBACK_CURSOR_POS]()
            pass

    def cursor_right(self, n=1):
        """ESCnC CUF (Cursor Forward)"""
        if not n:
            n = 1
        n = int(n)
        self.cursorX += 1
        try:
            pass

    def cursor_up(self, n=1):
        """ESCnA CUU (Cursor Up)"""
        if not n:
            n = 1
        n = int(n)
        self.cursorY = max(0, self.cursorY - n)
        try:
            pass

    def cursor_down(self, n=1):
        """ESCnB CUD (Cursor Down)"""
        if not n:
            n = 1
        n = int(n)
        self.cursorY = min(self.rows, self.cursorY + n)
        try:
            pass

    def cursor_next_line(self, n):
        """ESCnE CNL (Cursor Next Line)"""
        if not n:
            n = 1
        n = int(n)
        try:
            pass

    def cursor_previous_line(self, n):
        """ESCnF CPL (Cursor Previous Line)"""
        if not n:
            n = 1
        n = int(n)
        try:
            pass

    def cursor_horizontal_absolute(self, n):
        """ESCnG CHA (Cursor Horizontal Absolute)"""
        if not n:
            n = 1
        n = int(n)
        self.cursorX = n - 1 # -1 because cols is 0-based
        try:
            pass

    def cursor_position(self, coordinates):
        """
        ESCnH CUP (Cursor Position).  Move the cursor to the given coordinates.

        *coordinates*: Should be something like, 'row;col' (1-based) but,
        'row', 'row;', and ';col' are also valid (assumes 1 on missing value).

        If coordinates is '', the cursor will be moved to the top left (1;1).
        """
        # NOTE: Since this is 1-based we have to subtract 1 from everything to
        #       match how we store these values internally.
        if not coordinates:
            row, col = 0, 0
        elif ';' in coordinates:
            row, col = coordinates.split(';')
        else:
            row = coordinates
            col = 0
        try:
            row = int(row)
            row = 0
        try:
            col = int(col)
            col = 0
        # These ensure a positive integer while reducing row and col by 1:
        row = max(0, row - 1)
        col = max(0, col - 1)
        self.cursorY = row
        self.cursorX = col
        try:
            pass

    def cursor_position_vertical(self, n):
        """
        Vertical Line Position Absolute (VPA) - Moves the cursor to given line.
        """
        n = int(n)
        self.cursorY = n - 1

    def clear_screen(self):
        """
        Clears the screen.  Also used to emulate a terminal reset.
        """

    def clear_screen_from_cursor_down(self):
        """
        Clears the screen from the cursor down (Esc[J or Esc[0J).
        """
        self.screen[self.cursorY:] = [
           [u' ' for a in xrange(self.cols)] for a in self.screen[self.cursorY:]
        ]
        self.renditions[self.cursorY:] = [
           [None for a in xrange(self.cols)] for a in self.screen[self.cursorY:]
        ]

    def clear_screen_from_cursor_up(self):
        """
        Clears the screen from the cursor up (Esc[1J).
        """
        self.screen[:self.cursorY+1] = [
           [u' ' for a in xrange(self.cols)] for a in self.screen[:self.cursorY]
        ]
        self.renditions[:self.cursorY+1] = [
           [None for a in xrange(self.cols)] for a in self.screen[:self.cursorY]
        ]

    def clear_screen_from_cursor(self, n):
        """
        CSI*n*J ED (Erase Data).  This escape sequence uses the following rules::

            Esc[J   Clear screen from cursor down   ED0
            Esc[0J  Clear screen from cursor down   ED0
            Esc[1J  Clear screen from cursor up     ED1
            Esc[2J  Clear entire screen             ED2
        """
        try:
            n = int(n)
        except ValueError: # Esc[J
            n = 0
        clear_types = {
            0: self.clear_screen_from_cursor_down,
            1: self.clear_screen_from_cursor_up,
            2: self.clear_screen
        }
        try:
            clear_types[n]()
        except KeyError:
            print("Error: Unsupported number for escape sequence J")
        # Execute our callbacks
        try:
            pass
        try:
            pass

    def clear_line_from_cursor_right(self):
        """
        Clears the screen from the cursor right (Esc[K or Esc[0K).
        """
        self.screen[self.cursorY][self.cursorX:] = [
            u' ' for a in self.screen[self.cursorY][self.cursorX:]]
        # NOTE: We have to check if a rendition was just set for this cursor
        #       position since \x1b[K is only supposed to clear renditions that
        #       were set prior to the cursor being placed at its current
        #       position (which is odd--and I can't find any documentation that
        #       says that's how it is supposed to work but it seems to be the
        #       case in real-world testing).
        if self.prev_esc_buffer.endswith('m'): # Was a rendition, preserve it
            self.renditions[self.cursorY][self.cursorX+1:] = [
                None for a in self.screen[self.cursorY][self.cursorX:]]
        else: # Reset the cursor position's rendition to the end of the line
            self.renditions[self.cursorY][self.cursorX:] = [

    def clear_line_from_cursor_left(self):
        """
        Clears the screen from the cursor left (Esc[1K).
        """
        saved = self.screen[self.cursorY][self.cursorX:]
        saved_renditions = self.renditions[self.cursorY][self.cursorX:]
        self.screen[self.cursorY] = [
            u' ' for a in self.screen[self.cursorY][:self.cursorX]
        ] + saved
        self.renditions[self.cursorY] = [
            None for a in self.screen[self.cursorY][:self.cursorX]
        ] + saved_renditions

    def clear_line(self):
        """
        Clears the entire line (Esc[2K).
        """
        self.screen[self.cursorY] = [u' ' for a in xrange(self.cols)]
        self.renditions[self.cursorY] = [None for a in xrange(self.cols)]

    def clear_line_from_cursor(self, n):
        """
        CSI*n*K EL (Erase in Line).  This escape sequence uses the following
        rules::

            Esc[K   Clear screen from cursor right  EL0
            Esc[0K  Clear screen from cursor right  EL0
            Esc[1K  Clear screen from cursor left   EL1
            Esc[2K  Clear entire line               ED2
        """
        try:
            n = int(n)
            n = 0
            0: self.clear_line_from_cursor_right,
            1: self.clear_line_from_cursor_left,
            2: self.clear_line
        }
        try:
            print("Error: Unsupported number for CSI escape sequence K")
        try:
            pass
        try:
            pass

    def set_led_state(self, n):
        """
        Sets the values the dict, self.leds depending on *n* using the following
        rules:

            Esc[0q  Turn off all four leds  DECLL0
            Esc[1q  Turn on LED #1          DECLL1
            Esc[2q  Turn on LED #2          DECLL2
            Esc[3q  Turn on LED #3          DECLL3
            Esc[4q  Turn on LED #4          DECLL4
        """
        states = n.split(';')
        for state in states:
            state = int(state)
            if state == 0:
                self.leds[1] = False
                self.leds[2] = False
                self.leds[3] = False
                self.leds[4] = False
            else:
                self.leds[state] = True

    def __reduce_renditions(self, renditions):
        """
        Takes a list, *renditions*, and reduces it to its logical equivalent (as
        far as renditions go).  Example:

            [0, 32, 0, 34, 0, 32]

        Would become:

            [0, 32]

        Other Examples:

            [0, 1, 36, 36] -> [0, 1, 36]
            [0, 30, 42, 30, 42] -> [0, 30, 42]
            [36, 32, 44, 42] -> [32, 42]
        """
        out_renditions = []
        foreground = None
        background = None
        for i, rend in enumerate(renditions):
            if rend < 29:
                if rend not in out_renditions:
                    out_renditions.append(rend)
            elif rend > 29 and rend < 40:
                foreground = rend
            elif rend > 39 and rend < 50:
                background = rend
            else:
                out_renditions.append(rend)
        if foreground:
            out_renditions.append(foreground)
        if background:
            out_renditions.append(background)
        return out_renditions

    def _set_rendition(self, n):
        """
        Sets self.renditions[self.cursorY][self.cursorX] equal to n.split(';').
        *n* is expected to be a string of ECMA-48 rendition numbers separated by
        semicolons.  Example:
            '0;1;31'
        ...will result in:
            [0, 1, 31]

        Note that the numbers were converted to integers and the order was
        preserved.
        """
        # TODO: Make this whole thing faster (or prove it isn't possible).
        cursorY = self.cursorY
        cursorX = self.cursorX
        if cursorX >= self.cols: # We're at the end of the row
            return
            #cursorY += 1 # Set the rendition for the next line
            #cursorX = 0
        if cursorY >= self.rows:
            return # Don't bother setting renditions past the bottom
        if not n: # or \x1b[m (reset)
            self.renditions[cursorY][cursorX] = [0]
            return # No need for further processing; save some CPU
        # Convert the string (e.g. '0;1;32') to a list (e.g. [0,1,32]
        new_renditions = [int(a) for a in n.split(';') if a != '']
        if self.renditions[cursorY][cursorX] == new_renditions:
            return # Nothing left to do
        if self.renditions[cursorY][cursorX]:
            # Existing rendition for this spot... Tack it on the end
            self.renditions[cursorY][cursorX] += new_renditions
            # Remove duplicates and unnecessary renditions
            self.renditions[cursorY][cursorX] = self.__reduce_renditions(
                self.renditions[cursorY][cursorX])
        else:
            self.renditions[cursorY][cursorX] = self.__reduce_renditions(new_renditions)

    def __opt_handler(self, chars):
        """
        Optional special escape sequence handler for sequences matching
        RE_OPT_SEQ.  If self.CALLBACK_OPT is defined it will be called like so:
            self.CALLBACKS[self.CALLBACK_OPT](chars)

        Applications can use this escape sequence to define whatever special
        handlers they like.  It works like this: If an escape sequence is
        encountered matching RE_OPT_SEQ this method will be called with the
        inbetween *chars* (e.g. \x1b]_;<chars>\x07) as the argument.

        Applications can then do what they wish with *chars*.

        NOTE: I added this functionality so that plugin authors would have a
        mechanism to communicate with terminal applications.  See the SSH plugin
        for an example of how this can be done (there's channels of
        communication amongst ssh_connect.py, ssh.js, and ssh.py).
        """
        try:
            self.callbacks[self.CALLBACK_OPT](chars)
            # High likelyhood that nothing is defined.  No biggie.
            pass

    def __spanify_line(self, lines, renditions):
        """
        Iterates over *line* and *renditions*, applying HTML markup (spans)
        where appropriate and returns a single line as the result. It also marks
        the cursor position via a <span> tag at the appropriate location.
        """
        linecount = self.__linecounter.next()
        outline = ""
        charcount = 0
        rendition_classes = RENDITION_CLASSES
        classcount = 0
        spancount = 0
        current_classes = []
        foregrounds = ('f0','f1','f2','f3','f4','f5','f6','f7')
        backgrounds = ('b0','b1','b2','b3','b4','b5','b6','b7')
        for char, rendition in izip(lines, renditions):
            if char in "<>": # Have to convert lt/gt to HTML entities
                char = char.replace('<', '&lt;')
                char = char.replace('>', '&gt;')
            if rendition != None:
                classes = imap(rendition_classes.get, rendition)
                same_as_before = True
                for _class in classes:
                    if _class not in current_classes:
                        # Something changed...  Start a new span
                        same_as_before = False
                        if spancount:
                            outline += "</span>"
                            spancount -= 1
                        if 'reset' in _class:
                            if _class == 'reset':
                                current_classes = []
                            else:
                                reset_class = _class.split('reset')[0]
                                if reset_class == 'foreground':
                                    # Remove any foreground classes
                                    [current_classes.pop(i) for i, a in
                                     enumerate(current_classes) if a in
                                     foregrounds
                                    ]
                                elif reset_class == 'background':
                                     backgrounds
                                    ]
                                else:
                                    try:
                                        current_classes.remove(reset_class)
                                    except ValueError:
                                        # Trying to reset something that was
                                        # never set.  Ignore
                                        pass
                        else:
                            if _class in foregrounds:
                                [current_classes.pop(i) for i, a in
                                 enumerate(current_classes) if a in
                                 foregrounds
                                ]
                            elif _class in backgrounds:
                                 backgrounds
                                ]
                            current_classes.append(_class)
                if current_classes and not same_as_before:
                    outline += '<span class="%s">' % " ".join(current_classes)
                    spancount += 1
            if linecount == cursorY and charcount == cursorX: # Cursor position
                if self.show_cursor:
                    outline += '<span class="cursor">%s</span>' % char
                else:
                    outline += char
                # Use the simpler self.__spanify_line_no_cursor as a speedup:
                self.__temp = self.__spanify_line
                self.__spanify_line = self.__spanify_line_no_cursor
            else:
                outline += char
            charcount += 1
        for whatever in xrange(spancount):
            outline += "</span>"
        return outline

    def __spanify_line_no_cursor(self, lines, renditions):
        """
        Iterates over *lines* and *renditions*, applying HTML markup (spans)
        where appropriate and returns a single line as the result.
        """
            if char in "<>": # Have to escape lt/gt
                            else:
                                    ]
                                    ]
                                else:
                                    try:
                                        pass
                        else:
                                ]
                            elif _class in foregrounds:
                                ]
                if current_classes:
            outline += char

    def dump_html(self):
        """
        Dumps the terminal screen as a list of HTML-formatted lines.

        Note: This places <span class="cursor">(current character)</span> around
        the cursor location.
        """
        results = map(self.__spanify_line, self.screen, self.renditions)
        self.__reset_linecounter()
        # Reset self.__spanify_line back to default:
        try:
            self.__spanify_line = self.__temp
            del self.__temp
        except AttributeError:
            pass # No cursor yet
        scrollback = []
        if self.scrollback_buf:
            # Process the scrollback buffer too
            scrollback = map(
                self.__spanify_line_no_cursor, # Slightly faster
                self.scrollback_buf,
                self.scrollback_renditions
            )
        # Empty the scrollback buffer:
        return (scrollback, results)

    def dump_plain(self):
        """
        Dumps the screen and the scrollback buffer as-is then empties the
        scrollback buffer.
        """
        screen = self.screen
        scrollback = self.scrollback_buf
        return (scrollback, screen)

    def dump_components(self):
        """
        Dumps the screen and renditions as-is, the scrollback buffer as HTML,
        and the current cursor coordinates.  Also, empties the scrollback buffer

        NOTE: Was used in some performance-related experiments but might be
        useful for other patterns in the future so I've left it here.
        """
        screen = [a.tounicode() for a in self.screen]
            # Process the scrollback buffer into HTML
                self.__spanify_line_no_cursor, # Slightly faster (no cursor logic)
            )
        return (scrollback, screen, self.renditions, self.cursorY, self.cursorX)

    def dump(self):
        """
        Returns self.screen as a list of strings with no formatting.
        No scrollback buffer.  No renditions.

        NOTE: Does not empty the scrollback buffer.  Primarily used to get a
        quick glance of what is being displayed (when debugging).
        """
        out = []
        for line in self.screen:
            out.append("".join(line))
        return out

#
#
# For license information see LICENSE.txt

Gate One utility functions and classes.
"""

# Meta
__version__ = '1.0'
__version_info__ = (1, 0)

import os
import signal
import sys
import re
import errno
import base64
import uuid
import logging
import syslog
import mimetypes
from commands import getstatusoutput
from datetime import datetime, timedelta

# Globals
# This matches JUST the PIDs from the output of the pstree command
RE_PSTREE = re.compile(r'\(([0-9]*)\)')
# This is used by the raw() function to show control characters
REPLACEMENT_DICT = {
    0: u'^@',
    1: u'^A',
    2: u'^B',
    3: u'^C',
    4: u'^D',
    5: u'^E',
    6: u'^F',
    7: u'^G',
    8: u'^H',
    9: u'^I',
    10: u'^J',
    11: u'^K',
    12: u'^L',
    13: u'^M',
    14: u'^N',
    15: u'^O',
    16: u'^P',
    17: u'^Q',
    18: u'^R',
    19: u'^S',
    20: u'^T',
    21: u'^U',
    22: u'^V',
    23: u'^W',
    24: u'^X',
    25: u'^Y',
    26: u'^Z',
    27: u'^[',
    28: u'^\\',
    29: u'^]',
    30: u'^^',
    31: u'^_',
    127: u'^?',
}
# Syslog string-to-int dict used by string_to_syslog_facility()
FACILITIES = {
    'kern': syslog.LOG_KERN,
    'user': syslog.LOG_USER,
    'mail': syslog.LOG_MAIL,
    'daemon': syslog.LOG_DAEMON,
    'auth': syslog.LOG_AUTH,
    'syslog': syslog.LOG_SYSLOG,
    'lpr': syslog.LOG_LPR,
    'news': syslog.LOG_NEWS,
    'uucp': syslog.LOG_UUCP,
    'cron': syslog.LOG_CRON,
    'local0': syslog.LOG_LOCAL0,
    'local1': syslog.LOG_LOCAL1,
    'local2': syslog.LOG_LOCAL2,
    'local3': syslog.LOG_LOCAL3,
    'local4': syslog.LOG_LOCAL4,
    'local5': syslog.LOG_LOCAL5,
    'local6': syslog.LOG_LOCAL6,
    'local7': syslog.LOG_LOCAL7
}

# Exceptions
class UnknownFacility(Exception):
    """
    Raised if string_to_syslog_facility() is given a string that doesn't match
    a known syslog facility.
    """
    pass

class MimeTypeFail(Exception):
    """
    Raised by create_data_uri() if the mimetype of a file could not be guessed.
    """
    pass

# Functions
def noop():
    'Do nothing (i.e. "No Operation")'
    pass

def gen_self_signed_ssl(notAfter=None):
    """
    This method will generate a secure self-signed SSL key/certificate pair
    saving the result as 'certificate.pem' and 'keyfile.pem' in the current
    working directory.  By default the certificate will be valid for 10 years
    but this can be overridden by passing a valid timestamp via the *notAfter*
    argument.

    Examples::

        gen_self_signed_ssl(60 * 60 * 24 * 365) # 1-year certificate
        gen_self_signed_ssl() # 10-year certificate
    """
    import OpenSSL, time, random, sys
    pkey = OpenSSL.crypto.PKey()
    pkey.generate_key(OpenSSL.crypto.TYPE_RSA, 2048)
    # Save the key as 'keyfile.pem':
    f = open('keyfile.pem', 'w')
    f.write(OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, pkey))
    f.close()

    cert = OpenSSL.crypto.X509()
    cert.set_serial_number(random.randint(0, sys.maxint))
    cert.gmtime_adj_notBefore(0)
    if notAfter:
        cert.gmtime_adj_notAfter(notAfter)
    else:
        cert.gmtime_adj_notAfter(60 * 60 * 24 * 3650)
    cert.get_subject().CN = '*'
    cert.get_subject().O = 'Gate One Certificate'
    cert.get_issuer().CN = 'Untrusted Authority'
    cert.get_issuer().O = 'Self-Signed'
    cert.set_pubkey(pkey)
    cert.sign(pkey, 'md5')
    f = open('certificate.pem', 'w')
    f.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
    f.close()

def none_fix(val):
    """
    If *val* is a string meaning 'none', return None.  Otherwise just return
    *val* as-is.  Examples::

        >>> import utils
        >>> utils.none_fix('none')
        None
        >>> utils.none_fix('0')
        None
        >>> utils.none_fix('whatever')
        'whatever'
    """
    if isinstance(val, basestring) and val.lower() in ['none', '0', 'no']:
        return None
    else:
        return val

def str2bool(val):
    """
    Converts strings like, 'false', 'true', '0', and '1' into their boolean
    equivalents.  If no logical match is found, return False.  Examples::

        >>> utils.str2bool('false')
        False
        >>> utils.str2bool('1')
        True
        >>> utils.st2bool('whatever')
        False
    """
    if isinstance(val, basestring) and val.lower() in ['1', 'true', 'yes']:
        return True
    else:
        return False

def generate_session_id():
    """
    Returns a random, 45-character session ID.  Example::

        >>> utils.generate_session_id()
        'NzY4YzFmNDdhMTM1NDg3Y2FkZmZkMWJmYjYzNjBjM2Y5O'

    """
    return base64.b64encode(uuid.uuid4().hex + uuid.uuid4().hex)[:45]

def mkdir_p(path):
    """
    Pythonic version of "mkdir -p".  Example equivalents::

        >>> import commands, utils
        >>> utils.mkdir_p('/tmp/test/testing') # Does the same thing as below:
        >>> commands.getoutput('mkdir -p /tmp/test/testing')
    """
    try:
        os.makedirs(path)
    except OSError as exc: # Python >2.5
        if exc.errno == errno.EEXIST:
            pass
        else: raise

def cmd_var_swap(cmd, session=None, user_dir=None, user=None, time=None):
    """
    Returns *cmd* with special inline variables swapped out for their respective
    argument values.  The special variables are as follows:

        %SESSION% - *session*
        %USERDIR% - *user_dir*
        %USER% - *user*
        %TIME% - *time*

    This allows for unique or user-specific values to be swapped into command
    line arguments like so:

        ssh_connect.py -M -S '/tmp/%SESSION%/%r@%h:%p'
    """
    if session:
        cmd = cmd.replace(r'%SESSION%', session)
    if user_dir:
        cmd = cmd.replace(r'%USERDIR%', user_dir)
    if user:
        cmd = cmd.replace(r'%USER%', user)
    if time:
        cmd = cmd.replace(r'%TIME%', str(time))
    return cmd

def kill_dtached_proc(session, term):
    """
    Kills the dtach session associated with the given *term* and all its
    sub-processes.  Requires *session* so it can figure out the right
    processess to kill.
    """
    cmd = (
        "ps -ef | "
        "grep %s/dtach:%s | " # Limit to those matching our session/term combo
        "grep 'dtach -c' | " # Limit to the parent dtach process
        "grep -v grep | " # Get rid of grep from the results (if present)
        "awk '{print $2}'" % (session, term) # Just the PID please
    )
    retcode, pid = getstatusoutput(cmd)
    if pid:
        retcode, pstree = getstatusoutput('pstree -p %s' % pid)
        # pstree will look something like:
        #   dtach(19041)---python(19042)---ssh(19048)
        pids = RE_PSTREE.findall(pstree)
        # pids will be something like ['19041', '19042', '19048']
        for pid in pids:
            try:
                os.kill(int(pid), signal.SIGTERM)
            except OSError:
                pass # No biggie; child died with parent

def killall(session_dir):
    """
    Kills all running Gate One terminal processes including any detached dtach
    sessions.
    *session_dir* - The path to Gate One's session directory.
    """
    cmd = (
        "for i in `ls %s`; "
        "do ps -ef | grep $i | awk '{print $2}' | xargs kill; "
        "done"
        % session_dir
    )
    retcode, output = getstatusoutput(cmd)
    cmd = "rm -rf %s/*" % session_dir
    recode, output = getstatusoutput(cmd)

def create_plugin_static_links(static_dir, plugin_dir):
    """
    Creates symbolic links for all plugins in the ./static/ directory.  The
    equivalent of:

    .. ansi-block::

        \x1b[1;31mroot\x1b[0m@host\x1b[1;34m:~ $\x1b[0m ln -s *plugin_dir*/<plugin>/static *static_dir*/<plugin>

    This is so plugins can reference files in their static directories using the
    following straightforward path::

        https://<gate one>/static/<plugin name>/<some file>

    This function will also remove any dead links if a plugin is removed.
    """
    # Clean up dead links before we do anything else
    cmd = "find -L '%s' -type l -delete" % static_dir
    if retcode > 0:
        logging.error(
            "Error trying to clean up broken links in the static directory. "
            "Please check the permissions.\n\n%s" % output
        )
    # Create symbolic links for each plugin's respective static directory
    for directory in os.listdir(plugin_dir):
        plugin_name = directory
        directory = os.path.join(plugin_dir, directory) # Make absolute
        for f in os.listdir(directory):
            if f == 'static':
                abs_src_path = os.path.join(directory, f)
                abs_dest_path = os.path.join(static_dir, plugin_name)
                try:
                    os.symlink(abs_src_path, abs_dest_path)
                except OSError:
                    pass # Already exists

def get_plugins(plugin_dir):
    """
    Adds plugins' Python files to sys.path and returns a dictionary of
    JavaScript, CSS, and Python files contained in *plugin_dir* like so::

        {
            'js': [ # NOTE: These would be be inside *plugin_dir*/static
                '/static/happy_plugin/whatever.js',
                '/static/ssh/ssh.js',
            ],
            'css': ['/static/ssh/ssh.css'],
            'py': [ # NOTE: These will get added to sys.path
                'happy_plugin',
                'ssh'
            ],
        }

    \*.js files inside of *plugin_dir*/<the plugin>/static will get automatically
    added to Gate One's index.html like so:

    .. code-block:: html

        {% for jsplugin in jsplugins %}
            <script type="text/javascript" src="{{jsplugin}}"></script>
        {% end %}

    \*.css files will get added to the <head> like so:

        {% for cssplugin in cssplugins %}
            <link rel="stylesheet" href="{{cssplugin}}" type="text/css" media="screen" />
        {% end %}
    """
    out_dict = {'js': [], 'css': [], 'py': []}
        plugin = directory
        http_static_path = '/static/%s' % plugin
        plugin_files = os.listdir(directory)
        if "__init__.py" in plugin_files:
            out_dict['py'].append(plugin) # Just need the base
            sys.path.append(directory)
        else: # Look for .py files
            for plugin_file in plugin_files:
                if plugin_file.endswith('.py'):
                    plugin_path = os.path.join(directory, plugin_file)
                    sys.path.append(directory)
                    (basename, ext) = os.path.splitext(plugin_path)
                    basename = basename.split('/')[-1]
                    out_dict['py'].append(basename)
        for plugin_file in plugin_files:
            if plugin_file == 'static':
                static_dir = os.path.join(directory, plugin_file)
                for static_file in os.listdir(static_dir):
                    if static_file.endswith('.js'):
                        http_path = os.path.join(http_static_path, static_file)
                        out_dict['js'].append(http_path)
                    elif static_file.endswith('.css'):
                        out_dict['css'].append(http_path)
    # Sort all plugins alphabetically so the order in which they're applied can
    # be controlled somewhat predictably
    out_dict['py'].sort()
    out_dict['js'].sort()
    out_dict['css'].sort()
    return out_dict

def load_plugins(plugins):
    """
    Given a list of *plugins*, imports them.
    NOTE:  Assumes they're all in sys.path.
    """
    out_list = []
    for plugin in plugins:
        imported = __import__(plugin, None, None, [''])
        out_list.append(imported)
    return out_list

def merge_handlers(handlers):
    """
    Takes a list of Tornado *handlers* like this::

        [
            (r"/", MainHandler),
            (r"/ws", TerminalWebSocket),
            (r"/auth", AuthHandler),
            (r"/style", StyleHandler),
                ...
            (r"/style", SomePluginHandler),
        ]

    ...and returns a list with duplicate handlers removed; giving precedence to
    handlers with higher indexes.  This allows plugins to override Gate One's
    default handlers.  Given the above, this is what would be returned::

        [
                ...
        ]

    This example would replace the default "/style" handler with
    SomePluginHandler; overriding Gate One's default StyleHandler.
    """
    regexes = []
    handlers.reverse()
    for handler in handlers:
        if handler[0] not in regexes:
            regexes.append(handler[0])
            out_list.append(handler)
    out_list.reverse()

# NOTE: This function has been released under the Apache 2.0 license.
# See: http://code.activestate.com/recipes/577894-convert-strings-like-5d-and-60s-to-timedelta-objec/
def convert_to_timedelta(time_val):
    """
    Given a *time_val* (string) such as '5d', returns a timedelta object
    representing the given value (e.g. timedelta(days=5)).  Accepts the
    following '<num><char>' formats:

    =========   ======= ===================
    Character   Meaning Example
    s           Seconds '60s' -> 60 Seconds
    m           Minutes '5m'  -> 5 Minutes
    h           Hours   '24h' -> 24 Hours
    d           Days    '7d'  -> 7 Days

        >>> utils.convert_to_timedelta('7d')
        datetime.timedelta(7)
        >>> utils.convert_to_timedelta('24h')
        datetime.timedelta(1)
        >>> utils.convert_to_timedelta('60m')
        datetime.timedelta(0, 3600)
        >>> utils.convert_to_timedelta('120s')
        datetime.timedelta(0, 120)
    """
    num = int(time_val[:-1])
    if time_val.endswith('s'):
        return timedelta(seconds=num)
    elif time_val.endswith('m'):
        return timedelta(minutes=num)
    elif time_val.endswith('h'):
        return timedelta(hours=num)
    elif time_val.endswith('d'):
        return timedelta(days=num)

def process_opt_esc_sequence(chars):
    """
    Parse the *chars* passed from terminal.py by way of the special,
    optional escape sequence handler (e.g. '<plugin>|<text>') into a tuple of
    (<plugin name>, <text>).  Here's an example::

        >>> utils.process_opt_esc_sequence('ssh|user@host:22')
        ('ssh', 'user@host:22')
    """
    plugin = None
    text = ""
    try:
        plugin, text = chars.split('|')
    except Exception as e:
        pass # Something went horribly wrong!
    return (plugin, text)

def raw(text, replacement_dict=None):
    """
    Returns *text* as a string with special characters replaced by visible
    equivalents using *replacement_dict*.  If *replacement_dict* is None or
    False the global REPLACEMENT_DICT will be used.  Example::

        >>> test = '\\x1b]0;Some xterm title\x07'
        >>> print(utils.raw(test))
        '^[]0;Some title^G'
    """
    if not replacement_dict:
        replacement_dict = REPLACEMENT_DICT
    out = u''
        charnum = ord(char)
        if charnum in replacement_dict.keys():
            out += replacement_dict[charnum]
        else:
            out += char
    return out

def string_to_syslog_facility(facility):
    """
    Given a string (*facility*) such as, "daemon" returns the numeric
    syslog.LOG_* equivalent.
    """
    if facility.lower() in FACILITIES:
        return FACILITIES[facility.lower()]
    else:
        raise UnknownFacility(
            "%s does not match a known syslog facility" % repr(facility))

def create_data_uri(filepath):
    """
    Given a file at *filepath*, return that file as a data URI.

    Raises a MimeTypeFail exception if the mimetype could not be guessed.
    """
    mimetype = mimetypes.guess_type(filepath)[0]
    if not mimetype:
        raise MimeTypeFail("Could not guess mime type of: %s" % filepath)
    f = open(filepath).read()
    encoded = base64.b64encode(f).replace('\n', '')
    if len(encoded) > 65000:
        logging.warn(
            "WARNING: Data URI > 65,000 characters.  You're pushing it buddy!")
    data_uri = "data:%s;base64,%s" % (mimetype, encoded)
    return data_uri

#
#

# Meta

Authentication
==============
This module contains Gate One's authentication classes.  They map to Gate One's
--auth configuration option like so:

=============== ===================
--auth=none     NullAuthHandler
--auth=kerberos KerberosAuthHandler
--auth=google   GoogleAuthHandler

None or Anonymous
-----------------
By default Gate One will not authenticate users.  This means that user sessions
will be tied to their browser cookie and users will not be able to resume their
sessions from another computer/browser.  Most useful for situations where
session persistence and logging aren't important.

*All* users will show up as %anonymous using this authentication type.

.. note:: The % is there to avoid name conflicts.

Kerberos
--------
Kerberos authentication utilizes GSSAPI for Single Sign-on (SSO) but will fall
back to HTTP Basic authentication if GSSAPI auth fails.  This authentication
type can be integrated into any Kerberos infrastructure including Windows
Active Directory.

It is great for both transparent authentication and being able to tie sessions
and logs to specific users within your organization (compliance).

.. note:: The sso.py module itself has extensive documentation on this authentication type.

Google Authentication
---------------------
If you want persistent user sessions but don't care to run your own
authentication infrastructure this authentication type is for you.  Assuming,
of course, that your Gate One server and clients will have access to the
Internet.

.. note:: This authentication type is perfect if you're using Chromebooks (Chrome OS devices).

Docstrings
"""

import os

from utils import mkdir_p, generate_session_id

# 3rd party imports
import tornado.web
import tornado.auth
import tornado.escape

class BaseAuthHandler(tornado.web.RequestHandler):
    """The base class for all Gate One authentication handlers."""
    def get_current_user(self):
        """Tornado standard method--implemented our way."""
        user_json = self.get_secure_cookie("user")
        if not user_json: return None
        return tornado.escape.json_decode(user_json)

    def user_login(self, user):
        """
        Called immediately after a user authenticates successfully.  Saves
        session information in the user's directory.  Expects *user* to be a
        string containing the username or userPrincipalName. e.g. 'user@REALM'
        or just 'someuser'.
        """
        # Make a directory to store this user's settings/files/logs/etc
        user_dir = self.settings['user_dir'] + "/" + user
        logging.info("Creating user directory: %s" % user_dir)
        mkdir_p(user_dir)
        os.chmod(user_dir, 0700)
        session_file = user_dir + '/session'
        if os.path.exists(session_file):
            session_data = open(session_file).read()
            session_info = tornado.escape.json_decode(session_data)
        else:
            with open(session_file, 'w') as f:
                # Save it so we can keep track across multiple clients
                session_info = {
                    'go_upn': user, # FYI: UPN == userPrincipalName
                    'go_session': generate_session_id()
                }
                session_info_json = tornado.escape.json_encode(session_info)
                f.write(session_info_json)
        self.set_secure_cookie("user", tornado.escape.json_encode(session_info))

    def user_logout(self, user):
        """
        Called immediately after a user logs out.  Doesn't actually do
        anything.  Just potential future use at this point.
        """
        pass # Nothing here yet but someone might want to override it

class NullAuthHandler(BaseAuthHandler):
    """
    A handler for when no authentication method is chosen (i.e. --auth=none).
    """
    def get(self):
        """
        Sets the 'user' cookie with a new random session ID (*go_session*) and
        sets *go_upn* to '%anonymous'.
        """
        # % is valid on the filesystem but invalid for an actual username.
        # This ensures we won't have a conflict at some point with an actual
        # user.
        user = r'%anonymous'
        self.user_login(user) # Takes care of the user's settings dir
        user_cookie = {
            'go_upn': user,
            'go_session': generate_session_id()
        }
        self.set_secure_cookie("user", tornado.escape.json_encode(user_cookie))
        next_url = self.get_argument("next", None)
        if next_url:
            self.redirect(next_url)
        else:
            self.redirect("/")

class GoogleAuthHandler(BaseAuthHandler, tornado.auth.GoogleMixin):
    """
    Google authentication handler.
    """
    @tornado.web.asynchronous
        """
        Sets the 'user' cookie with an appropriate *go_upn* and *go_session*.
        """
        if self.get_argument("openid.mode", None):
            self.get_authenticated_user(self._on_auth)
            return
        self.authenticate_redirect(
            ax_attrs=["name","email","language","username"])

    def _on_auth(self, user):
        """
        Just a continuation of the get() method (the final step where it
        actually sets the cookie).
        """
        if not user:
            raise tornado.web.HTTPError(500, "Google auth failed")
        print("GoogleAuthLoginHandler user: %s" % user)
        # NOTE: Google auth 'user' will be a dict like so:
        # user: {
        #     'locale': u'en-us',
        #     'first_name': u'Dan',
        #     'last_name': u'McDougall',
        #     'name': u'Dan McDougall',
        #     'email': u'riskable@gmail.com'}
        # Named these 'go_<whatever>' since that is less likely to conflict with
        # anything in the future (should some auth mechanism start returning
        # session IDs of some sort).
        self.user_login(user) # This takes care of the user's settings dir
        user_cookie = { # Don't need all that other stuff
            'go_session': generate_session_id(),
            'go_upn': user['email'] # Just an equivalent for standardization
        }
        else:

# Add our KerberosAuthHandler if sso is available
KerberosAuthHandler = None
try:
    from sso import KerberosAuthMixin
    class KerberosAuthHandler(BaseAuthHandler, KerberosAuthMixin):
        """
        Handles authenticating users via Kerberos/GSSAPI/SSO.
        """
        def get(self):
            """
            Checks the user's request header for the proper Authorization data. If
            it checks out the user will be logged in via _on_auth().  If not, the
            browser will be redirected to login.
            """
            auth_header = self.request.headers.get('Authorization')
            if auth_header:
                self.get_authenticated_user(self._on_auth)
                return
            self.authenticate_redirect()

        def _on_auth(self, user):
            if not user:
                raise tornado.web.HTTPError(500, "Kerberos auth failed")
            self.user_login(user) # This takes care of the user's settings dir
            # TODO: Add some LDAP or local DB lookups here to add more detail to user objects
            logging.debug("KerberosAuthHandler user: %s" % user)
            next_url = self.get_argument("next", None)
            if next_url:
                self.redirect(next_url)
            else:
                self.redirect("/")
except ImportError:
    pass # No SSO available.

#
#

# 1.0 TODO:
# * DOCUMENTATION!
# * Write a setup.py' with init scripts to stop/start/restart Gate One safely.  Also make sure that .deb and .rpm packages safely restart Gate One without impacting running sessions.  The setup.py should also attempt to minify the .css and .js files.

# Meta

# NOTE: Docstring includes reStructuredText markup for use with Sphinx.
__doc__ = '''\
.. _gateone.py:

Gate One
========
Gate One is a web-based terminal emulator written in Python using the Tornado
web framework.  This module runs the primary daemon process and acts as a
central controller for all running terminals and terminal programs.  It supports
numerous configuration options and can also be called with the --kill switch
to kill all running terminal programs (if using dtach--otherwise they die on
their own when gateone.py is stopped).

Dependencies
------------
Gate One requires Python 2.6+ but runs best with Python 2.7+.  It also depends
on the following 3rd party Python modules:

 * `Tornado <http://www.tornadoweb.org/>`_ 2.1+ - Non-blocking web server framework that powers FriendFeed.

The following modules are optional and can provide Gate One with additional
functionality:

 * `pyOpenSSL <https://launchpad.net/pyopenssl>`_ 0.10+ - OpenSSL wrapper for Python.  Only used to generate self-signed SSL keys and certificates.
 * `kerberos <http://pypi.python.org/pypi/kerberos>`_ 1.0+ - A high-level Kerberos interface for Python.  Only necessary if you plan to use the Kerberos authentication module.

On most platforms both the required and optional modules can be installed via one of these commands:

        \x1b[1;34muser\x1b[0m@modern-host\x1b[1;34m:~ $\x1b[0m sudo pip install tornado pyopenssl kerberos

...or:

        \x1b[1;34muser\x1b[0m@legacy-host\x1b[1;34m:~ $\x1b[0m sudo easy_install tornado pyopenssl kerberos

.. note:: The use of pip is recommended.  See http://www.pip-installer.org/en/latest/installing.html if you don't have it.

Settings
--------
All of Gate One's configurable options can be controlled either via command line
switches or by settings in the server.conf file (they match up 1-to-1).  If no
server.conf exists one will be created using defaults (i.e. when Gate One is run
for the first time).  Settings in the server.conf file use the following format::

    <setting> = <value>

Here's an example::

    address = "0.0.0.0" # Strings are surrounded by quotes
    port = 443 # Numbers don't need quotes

There are a few important differences between the configuration file and
command line switches in regards to boolean values (True/False).  A switch such
as --debug evaluates to "debug = True" and this is exactly how it would be
configured in server.conf::

    debug = True # Booleans don't need quotes either

.. note:: server.conf is case sensitive for "True", "False" and "None".

Running gateone.py with the --help switch will print the usage information as
well as descriptions of what each configurable option does:

    \x1b[1;31mroot\x1b[0m@host\x1b[1;34m:~ $\x1b[0m ./gateone.py --help
    Usage: ./gateone.py [OPTIONS]

    Options:
      --help                           show this help information
      --log_file_max_size              max size of log files before rollover
      --log_file_num_backups           number of log files to keep
      --log_file_prefix=PATH           Path prefix for log files. Note that if you are running multiple tornado processes, log_file_prefix must be different for each of them (e.g. include the port number)
      --log_to_stderr                  Send log output to stderr (colorized if possible). By default use stderr if --log_file_prefix is not set and no other logging is configured.
      --logging=info|warning|error|none Set the Python log level. If 'none', tornado won't touch the logging configuration.
      --address                        Run on the given address.
      --auth                           Authentication method to use.  Valid options are: none, kerberos, google
      --certificate                    Path to the SSL certificate.  Will be auto-generated if none is provided.
      --command                        Run the given command when a user connects (e.g. 'nethack').
      --cookie_secret                  Use the given 45-character string for cookie encryption.
      --debug                          Enable debugging features such as auto-restarting when files are modified.
      --disable_ssl                    If enabled, Gate One will run without SSL (generally not a good idea).
      --dtach                          Wrap terminals with dtach. Allows sessions to be resumed even if Gate One is stopped and started (which is a sweet feature =).
      --embedded                       Run Gate One in Embedded Mode (no toolbar, only one terminal allowed, etc.  See docs).
      --keyfile                        Path to the SSL keyfile.  Will be auto-generated if none is provided.
      --kill                           Kill any running Gate One terminal processes including dtach'd processes.
      --port                           Run on the given port.
      --session_dir                    Path to the location where session information will be stored.
      --session_logging                If enabled, logs of user sessions will be saved in <user_dir>/logs.  Default: Enabled
      --session_timeout                Amount of time that a session should be kept alive after the client has logged out.  Accepts <num>X where X could be one of s, m, h, or d for seconds, minutes, hours, and days.  Default is '5d' (5 days).
      --sso_realm                      Kerberos REALM (aka DOMAIN) to use when authenticating clients. Only relevant if Kerberos authentication is enabled.
      --sso_service                    Kerberos service (aka application) to use. Defaults to HTTP. Only relevant if Kerberos authentication is enabled.
      --syslog_facility                Syslog facility to use when logging to syslog (if syslog_session_logging is enabled).  Must be one of: auth, cron, daemon, kern, local0, local1, local2, local3, local4, local5, local6, local7, lpr, mail, news, syslog, user, uucp.  Default: daemon
      --syslog_session_logging         If enabled, logs of user sessions will be written to syslog.
      --user_dir                       Path to the location where user files will be stored.

.. note:: Some of these options (e.g. log_file_prefix) are inherent to the Tornado framework.  You won't find them anywhere in gateone.py.

File Paths
----------
Gate One stores its files, temporary session information, and persistent user
data in the following locations (Note: Many of these are configurable):

================= ==================================================================================
File/Directory      Description
gateone.py        Gate One's primary executable/script. Also, the file containing this documentation
auth.py           Authentication classes
logviewer.py      A utility to view Gate One session logs
server.conf       Gate One's configuration file
sso.py            A Kerberos Single Sign-on module for Tornado (used by auth.py)
terminal.py       A Pure Python terminal emulator module
termio.py         Terminal input/output control module
utils.py          Various supporting functions
docs/             Gate One documentation
static/           Non-dynamic files that get served to clients (e.g. gateone.js, gateone.css, etc)
templates/        Tornado template files such as index.html
tests/            Gate One-specific automated unit/acceptance tests
plugins/          Plugins go here in the form of ./plugins/<plugin name>/<plugin files|directories>
users/            Persistent user data in the form of ./users/<username>/<user-specific files>
users/<user>/logs This is where session logs get stored if session_logging is set.
/tmp/gateone      Temporary session data in the form of /tmp/gateone/<session ID>/<files>

Running
-------
Executing Gate One is as simple as:

    \x1b[1;31mroot\x1b[0m@host\x1b[1;34m:~ $\x1b[0m ./gateone.py

NOTE: By default Gate One will run on port 443 which requires root on most
systems.  Use --port=<something greater than 1024> for non-root users.

Plugins
-------
Gate One includes support for any combination of the following types of plugins:

 * Python
 * JavaScript
 * CSS

Python plugins can integrate with Gate One in three ways:

 * Adding or overriding tornado.web.RequestHandlers (with a given regex).
 * Adding or overriding methods (aka "commands") in TerminalWebSocket.
 * Adding special plugin-specific escape sequence handlers (see the plugin development documentation for details on what/how these are/work).

JavaScript plugins will be added to the <body> tag of Gate One's base index.html
template like so:

.. code-block:: html

    <!-- Begin JS files from plugins -->
    {% for jsplugin in jsplugins %}
    <script type="text/javascript" src="{{jsplugin}}"></script>
    {% end %}
    <!-- End JS files from plugins -->

CSS plugins are similar to JavaScript but instead of being appended to the
<body> they are added to the <head>:

    <!-- Begin CSS files from plugins -->
    {% for cssplugin in cssplugins %}
    <link rel="stylesheet" href="{{cssplugin}}" type="text/css" media="screen" />
    {% end %}
    <!-- End CSS files from plugins -->

There are also hooks throughout Gate One's code for plugins to add or override
Gate One's functionality.  Documentation on how to write plugins can be found in
the Plugin Development docs.  From the perspective of gateone.py, it performs
the following tasks in relation to plugins:

 * Imports Python plugins and connects their hooks.
 * Creates symbolic links inside ./static/ that point to each plugin's respective /static/ directory and serves them to clients.
 * Serves the index.html that includes plugins' respective .js and .css files.

'''

# Standard library modules
import os
import sys
import threading
import time
from functools import partial
from platform import uname

# Our own modules
import termio
from auth import NullAuthHandler, KerberosAuthHandler, GoogleAuthHandler
from utils import noop, str2bool, generate_session_id, cmd_var_swap, mkdir_p
from utils import gen_self_signed_ssl, killall, get_plugins, load_plugins
from utils import create_plugin_static_links, merge_handlers, none_fix
from utils import convert_to_timedelta, kill_dtached_proc
from utils import process_opt_esc_sequence, create_data_uri
from utils import FACILITIES, string_to_syslog_facility

# Tornado modules (yeah, we use all this stuff)
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.template
from tornado.websocket import WebSocketHandler
from tornado.escape import json_encode, json_decode
from tornado.options import define, options

# Globals
SESSIONS = {} # We store the crux of most session info here
CMD = None # Will be overwritten by options.command
TIMEOUT = timedelta(days=5) # Gets overridden by options.session_timeout
GATEONE_DIR = os.path.dirname(os.path.abspath(__file__))
PLUGINS = get_plugins(GATEONE_DIR + '/plugins')
PLUGIN_WS_CMDS = {} # Gives plugins the ability to extend/enhance TerminalWebSocket
PLUGIN_HOOKS = {} # Gives plugins the ability to hook into various things.
# Gate One registers a handler for for terminal.py's CALLBACK_OPT special escape
# sequence callback.  Whenever this escape sequence is encountered, Gate One
# will parse the sequence's contained characters looking for the following
# format:
#   <plugin name>|<whatever>
# The <whatever> part will be passed to any plugin matching <plugin name> if the
# plugin has 'Escape': <function> registered in its hooks.
PLUGIN_ESC_HANDLERS = {}

# Classes
class BaseHandler(tornado.web.RequestHandler):
    """
    A base handler that all Gate One RequestHandlers will inherit methods from.
    """
    # Right now it's just the one...

class MainHandler(BaseHandler):
    """
    Renders index.html which loads Gate One.

    Will include the minified version of gateone.js if available as
    gateone.min.js.

    Will encode GATEONE_DIR/static/bell.ogg as a data:URI and put it as the
    <source> of the <audio> tag inside the index.html template.  Gate One
    administrators can replace bell.ogg with whatever they like but the file
    size should be less than 32k when encoded to Base64.
    """
    # TODO: Add the ability for users to define their own individual bells.
    @tornado.web.authenticated
        hostname = uname()[1]
        gateone_js = "/static/gateone.js"
        minified_js_abspath = "%s/static/gateone.min.js" % GATEONE_DIR
        bell = "%s/static/bell.ogg" % GATEONE_DIR
        bell_data_uri = create_data_uri(bell)
        # Use the minified version if it exists
        if os.path.exists(minified_js_abspath):
            gateone_js = "/static/gateone.min.js"
        self.render(
            "templates/index.html",
            hostname=hostname,
            gateone_js=gateone_js,
            jsplugins=PLUGINS['js'],
            cssplugins=PLUGINS['css'],
            bell_data_uri=bell_data_uri
        )

class StyleHandler(BaseHandler):
    """
    Serves up our CSS templates (e.g. the 'black' or 'white' schemes)
    """
    # Hey, if unauthenticated clients want this they can have it!
    #@tornado.web.authenticated
        container = self.get_argument("container")
        prefix = self.get_argument("prefix")
        scheme = self.get_argument("scheme", None)
        self.set_header ('Content-Type', 'text/css')
            "templates/css_%s.css" % scheme, container=container, prefix=prefix)

class TerminalWebSocket(WebSocketHandler):
    def __init__(self, application, request):
        WebSocketHandler.__init__(self, application, request)
        self.commands = {
            'ping': self.pong,
            'authenticate': self.authenticate,
            'new_terminal': self.new_terminal,
            'set_terminal': self.set_terminal,
            'kill_terminal': self.kill_terminal,
            'c': self.char_handler, # Just 'c' to keep the bandwidth down
            'refresh': self.refresh_screen,
            #'refresh_test': self.refresh_screen_testing,
            'resize': self.resize,
            'debug_terminal': self.debug_terminal
        }
        self.terms = {}
        # So we can keep track and avoid sending unnecessary messages:
        self.titles = {}

        """Identical to the function of the same name in MainHandler."""

    def open(self):
        """Called when a new WebSocket is opened."""
        logging.info(
            "WebSocket opened (%s)" % self.get_current_user()['go_upn'])

    def on_message(self, message):
        """Called when we receive a message from the client."""
        # This is super useful when debugging:
        logging.debug("message: %s" % repr(message))
        message_obj = None
        try:
            message_obj = json_decode(message) # JSON FTW!
            if not isinstance(message_obj, dict):
                self.write_message("'Error: Message bust be a JSON dict.'")
        except ValueError: # We didn't get JSON
            self.write_message("'Error: We only accept JSON here.'")
        if message_obj:
            for key, value in message_obj.items():
                try: # Plugins first so they can override behavior if they wish
                    PLUGIN_WS_CMDS[key](value, tws=self) # tws==TerminalWebSocket
                except KeyError:
                    try:
                        self.commands[key](value)
                        pass # Ignore commands we don't understand

    def on_close(self):
        """
        Called when the client terminates the connection.

        NOTE: Normally self.refresh_screen() catches the disconnect first and
        this won't be called.
        """
            "WebSocket closed (%s)" % self.get_current_user()['go_upn'])

    def pong(self, timestamp):
        """
        Responds to a client 'ping' request...  Just returns the given
        timestamp back to the client so it can measure round-trip time.
        """
        message = {'pong': timestamp}
        self.write_message(json_encode(message))

# TODO: Change this to encrypt the session ID so that it is stored in encrypted form on the client end.  Just like we do with cookies but for use with localStorage.  The encrypted value should actually be a JSON dict with a uniqe, random ID included to ensure that the encrypted data changes every time it is created (even though the session ID might not).
    def authenticate(self, settings):
        """
        Authenticates the client using the given session (which should be
        settings['session']) and returns a list of all running terminals (if
        any).  If no session is given (null) a new one will be created.
        """
        logging.debug("authenticate(): %s" % settings)
        # Make sure the client is authenticated if authentication is enabled
        if self.settings['auth']:
            try:
                user = self.get_current_user()['go_upn']
                if user == '%anonymous':
                    logging.error("Unauthenticated WebSocket attempt.")
                    # In case this is a legitimate client that simply lost its
                    # cookie, tell it to re-auth by calling the appropriate
                    # action on the other side.
                    message = {'reauthenticate': True}
                    self.write_message(json_encode(message))
                    self.close() # Close the WebSocket
            except KeyError:
                # Force them to authenticate
                message = {'reauthenticate': True}
                self.write_message(json_encode(message))
                self.close() # Close the WebSocket
        else:
            # Double-check there isn't a user set in the cookie (i.e. we have
            # recently changed Gate One's settings).  If there is, force it
            # back to %anonymous.
            user = self.get_current_user()['go_upn']
            if user != '%anonymous':
        if 'session' in settings.keys():
            # Try to use the cookie session first
            try:
                self.session = self.get_current_user()['go_session']
            except:
                # This generates a random 45-character string:
                self.session = generate_session_id()
        # This check is to make sure there's no existing session so we don't
        # accidentally clobber it.
        if self.session not in SESSIONS:
            # Old session is no good, start a new one:
            SESSIONS[self.session] = {}
        terminals = []
        for term in SESSIONS[self.session].keys():
            if isinstance(term, int):  # This skips the TidyThread...
                terminals.append(term) # Only terminals are integers in the dict
        # Check for any dtach'd terminals we might have missed
        if self.settings['dtach']:
            session_dir = self.settings['session_dir']
            session_dir = session_dir + "/" + self.session
            if not os.path.exists(session_dir):
                mkdir_p(session_dir)
                os.chmod(session_dir, 0700)
            for item in os.listdir(session_dir):
                if item.startswith('dtach:'):
                    term = int(item.split(':')[1])
                    if term not in terminals:
                        terminals.append(term)
        terminals.sort() # Put them in order so folks don't get confused
        message = {'terminals': terminals}
        # TODO: Add a hook here for plugins to send their own messages when a
        #       given terminal is reconnected.

    def new_terminal(self, settings):
        """
        Starts up a new terminal associated with the user's session using
        *settings* as the parameters.  If a terminal already exists with the
        same number as *settings[term]* self.set_terminal() will be called
        instead of starting a new terminal (so clients can resume their session
        without having to worry about figuring out if a new terminal already
        exists or not).
        """
        logging.debug("%s new_terminal(): %s" % (
            self.get_current_user()['go_upn'], settings))
        self.current_term = term = settings['term']
        self.rows = rows = settings['rows']
        self.cols = cols = settings['cols']
        user_dir = self.settings['user_dir']
        if term not in SESSIONS[self.session]:
            # Setup the requisite dict
            SESSIONS[self.session][term] = {}
        if 'multiplex' not in SESSIONS[self.session][term]:
            # Start up a new terminal
            SESSIONS[self.session][term]['created'] = datetime.now()
            # NOTE: Not doing anything with 'created'...  yet!
            now = int(round(time.time() * 1000))
            try:
            except:
                # No auth, use %anonymous (% is there to prevent conflicts)
                user = r'%anonymous' # Don't get on this guy's bad side
            cmd = cmd_var_swap(CMD,   # Swap out variables like %USER% in CMD
                session=self.session, # with their real-world values.
                user_dir=user_dir,
                user=user,
                time=now
            )
            resumed_dtach = False
            # Create the session dir if not already present
            if self.settings['dtach']: # Wrap in dtach (love this tool!)
                dtach_path = "%s/dtach:%s" % (session_dir, term)
                if os.path.exists(dtach_path):
                    # Using 'none' for the refresh because the EVIL termio
                    # likes to manage things like that on his own...
                    cmd = "dtach -a %s -E -z -r none" % dtach_path
                    resumed_dtach = True
                else: # No existing dtach session...  Make a new one
                    cmd = "dtach -c %s -E -z -r none %s" % (dtach_path, cmd)
            log_path = None
            if self.settings['session_logging']:
                log_dir = "%s/%s/logs" % (user_dir, user)
                # Create the log dir if not already present
                if not os.path.exists(log_dir):
                    mkdir_p(log_dir)
                log_path = "%s/%s" % (
                    log_dir, datetime.now().strftime('%Y%m%d%H%M%S%f.golog'))
            facility = string_to_syslog_facility(
                self.settings['syslog_facility'])
            SESSIONS[self.session][term]['multiplex'] = termio.Multiplex(
                cmd,
                tmpdir=session_dir,
                log_path=log_path,
                term_num=term,
                syslog=self.settings['syslog_session_logging'],
                syslog_facility=facility
            )
            # Set some environment variables so the programs we execute can use
            # them (very handy).  Allows for "tight integration" and "synergy"!
            env = {
                'GO_TERM': str(term),
                'GO_SESSION': self.session,
                'GO_SESSION_DIR': session_dir,
            }
            fd = SESSIONS[self.session][term]['multiplex'].create(
                rows, cols, env=env)
            refresh = partial(self.refresh_screen, term)
            SESSIONS[self.session][term][ # 1 is CALLBACK_UPDATE
                'multiplex'].callbacks[1] = refresh
            restart = partial(self.new_terminal, settings)
            SESSIONS[self.session][term][ # 2 is CALLBACK_EXIT
                'multiplex'].callbacks[2] = restart
            termio_write = SESSIONS[ # Write responses directly to the prog
                self.session][term]['multiplex'].proc_write
            SESSIONS[self.session][term][ # 5 is CALLBACK_DSR
                'multiplex'].term.callbacks[5] = termio_write
            set_title = partial(self.set_title, term)
            SESSIONS[self.session][term][ # 6 is CALLBACK_TITLE
                'multiplex'].term.callbacks[6] = set_title
            set_title() # Set initial title
            bell = partial(self.bell, term)
            SESSIONS[self.session][term][ # 7 is CALLBACK_BELL
                'multiplex'].term.callbacks[7] = bell
            SESSIONS[self.session][term][ # 8 is CALLBACK_OPT
                    'multiplex'].term.callbacks[8] = self.esc_opt_handler
            mode_handler = partial(self.mode_handler, term)
            SESSIONS[self.session][term][ # 9 is CALLBACK_MODE
                    'multiplex'].term.callbacks[9] = mode_handler
            if self.settings['dtach']: # dtach sessions need a little extra love
                SESSIONS[self.session][term]['multiplex'].redraw()
        else:
            # Terminal already exists
            if SESSIONS[self.session][term]['multiplex'].alive: # It's ALIVE!
                message = {'term_exists': term}
                # This resets the diff
                SESSIONS[self.session][term]['multiplex'].prev_output = [
                    None for a in xrange(rows-1)]
                # TODO: Right here we need to change how the callbacks are handled so we can have multiple screen update callbacks for the same session (so a user could have two browsers open to the same session).  Not exactly a common use case but you never know!
                restart = partial(self.new_terminal, settings)
                SESSIONS[self.session][term][ # 2 is CALLBACK_EXIT
                    'multiplex'].callbacks[2] = restart
                refresh = partial(self.refresh_screen, term)
                SESSIONS[self.session][term][ # 1 is CALLBACK_UPDATE
                    'multiplex'].callbacks[1] = refresh
                set_title = partial(self.set_title, term)
                termio_write = SESSIONS[ # Write responses directly to the prog
                    self.session][term]['multiplex'].proc_write
                SESSIONS[self.session][term][ # 5 is CALLBACK_DSR
                SESSIONS[self.session][term][ # 6 is CALLBACK_TITLE
                    'multiplex'].term.callbacks[6] = set_title
                set_title() # Set the title
                bell = partial(self.bell, term)
                SESSIONS[self.session][term][ # 7 is CALLBACK_BELL
                    'multiplex'].term.callbacks[7] = bell
                SESSIONS[self.session][term][ # 8 is CALLBACK_OPT
                mode_handler = partial(self.mode_handler, term)
                SESSIONS[self.session][term][ # 9 is CALLBACK_MODE
            self.refresh_screen(term) # Send a fresh screen to the client
            # NOTE: refresh_screen will also take care of cleaning things up if
            #       SESSIONS[self.session][term]['multiplex'].alive is False
        if 'tidy_thread' not in SESSIONS[self.session]:
            # Start the keepalive thread so the session will time out if the
            # user disconnects for like a week (by default anyway =)
            SESSIONS[self.session]['tidy_thread'] = TidyThread(self.session)
            SESSIONS[self.session]['tidy_thread'].start()

    def kill_terminal(self, term):
        """Kills *term* and any associated processes"""
        #print("killing terminal: %s" % term)
        term = int(term)
        try:
            SESSIONS[self.session][term]['multiplex'].die()
            SESSIONS[self.session][term]['multiplex'].proc_kill()
            if self.settings['dtach']:
                kill_dtached_proc(self.session, term)
            del SESSIONS[self.session][term]
        except KeyError as e:
            pass # The EVIL termio has killed my child!  Wait, that's good...
                 # Because now I don't have to worry about it!

    def set_terminal(self, term):
        """Sets self.current_term = *term*"""
        self.current_term = term

    def set_title(self, term):
        """
        Sends a message to the client telling it to set the window title of
        *term* to...
        SESSIONS[self.session][term]['multiplex'].proc[fd]['term'].title.

        Example output:

            {'set_title': {'term': 1, 'title': "user@host"}}
        """
        #print("Got set_title on term: %s" % term)
        title = SESSIONS[self.session][term]['multiplex'].term.title
        # Only send a title update if it actually changed
        if term not in self.titles: # There's a first time for everything
            self.titles[term] = ""
        if title != self.titles[term]:
            self.titles[term] = title
            title_message = {'set_title': {'term': term, 'title': title}}
            self.write_message(json_encode(title_message))

    def bell(self, term):
        """
        Sends a message to the client indicating that a bell was encountered in
        the given terminal (*term*).  Example output:

        {'bell': {'term': 1}}
        """
        bell_message = {'bell': {'term': term}}
        self.write_message(json_encode(bell_message))

    def mode_handler(self, term, setting, boolean):
        """Handles mode settings that require an action on the client."""
        if setting in ['1']: # Only support this mode right now
            if boolean:
                # Tell client to enable application cursor mode
                mode_message = {'set_mode': {
                    'mode': setting,
                    'boolean': True,
                    'term': term
                }}
                self.write_message(json_encode(mode_message))
            else:
                # Tell client to disable application cursor mode
                    'boolean': False,
                }}

    def refresh_screen(self, term):
        """Returns the whole terminal screen."""
        try:
            SESSIONS[self.session]['tidy_thread'].keepalive(datetime.now())
            scrollback, screen = SESSIONS[
                self.session][term]['multiplex'].dumplines()
        except KeyError: # Session died (i.e. command ended).
            scrollback, screen = None, None
        if screen:
            multiplexer = SESSIONS[self.session][term]['multiplex']
            output_dict = {
                'termupdate': {
                    'term': term,
                    'scrollback': scrollback,
                    'screen' : screen,
                    'ratelimiter': multiplexer.ratelimiter_engaged
                }
            }
            try:
                self.write_message(json_encode(output_dict))
            except IOError: # Socket was just closed, no biggie
                logging.info(
                    "WebSocket closed (%s)" % self.get_current_user()['go_upn'])
                    'multiplex'].callbacks[1] = noop # Stop trying to write

    def resize(self, resize_obj):
        """
        Resize the terminal window to the rows/cols specified in *resize_obj*

        Example *resize_obj*:
            {'rows': 24, 'cols': 80}
        """
        self.rows = resize_obj['rows']
        self.cols = resize_obj['cols']
        term = resize_obj['term']
        if self.rows < 2 or self.cols < 2: # 0 or negative numbers will crash
            # Fall back to a standard default:
            self.rows = 24
            self.cols = 80
        # If the user already has a running session, set the new terminal size:
        try: # TODO: Make this only resize a given terminal.  Let the client handle repeating the resize command for each.
            for term in SESSIONS[self.session].keys():
                if isinstance(term, int): # Skip the TidyThread
                    SESSIONS[self.session][term]['multiplex'].resize(
                        self.rows,
                        self.cols
                    )
        except KeyError: # Session doesn't exist yet, no biggie
            pass

    def char_handler(self, chars):
        """Writes *chars* (string) to the currently-selected terminal"""
        term = self.current_term
        session = self.session
        if session in SESSIONS:
            if SESSIONS[session][term]['multiplex'].alive:
                if chars:
                    SESSIONS[ # Force an update
                        session][term]['multiplex'].ratelimit = time.time()
                    SESSIONS[session][term]['multiplex'].proc_write(chars)

    def esc_opt_handler(self, chars):
        """
        Executes whatever function is registered matching the tuple returned by
        process_opt_esc_sequence().
        """
        plugin_name, text = process_opt_esc_sequence(chars)
        if plugin_name:
            try:
                PLUGIN_ESC_HANDLERS[plugin_name](text, tws=self)
            except Exception as e:
                print("Got exception trying to execute plugin's optional ESC "
                      "sequence handler...")
                print(e)

    def debug_terminal(self, term):
        """
        Prints the terminal's screen and renditions to stdout so they can be
        examined more closely.

        NOTE: Can only be called from a JavaScript console like so:
                GateOne.ws.send(JSON.stringify({'debug_terminal': *term*}));
        """
        screen = SESSIONS[self.session][term]['multiplex'].term.screen
        renditions = SESSIONS[self.session][term]['multiplex'].term.renditions
        for i, line in enumerate(screen):
            print("%s:%s" % (i, "".join(line)))
            print(renditions[i])

class RecordingHandler(BaseHandler):
    """
    Handles uploads of session recordings and returns them to the client in a
    self-contained HTML file that will auto-start playback.

    NOTE: The real crux of the code that handles this is in the template.
    """
    def post(self):
        recording = self.get_argument("recording")
        scheme = self.get_argument("scheme")
        css_file = open('templates/css_%s.css' % scheme).read()
        css = tornado.template.Template(css_file)
            "templates/self_contained_recording.html",
            recording=recording,
            container=container,
            prefix=prefix,
            css=css.generate(container=container, prefix=prefix)
        )

class OpenLogHandler(BaseHandler):
    """
    Handles uploads of user logs and returns them to the client as a basic HTML
    page.  Essentially, this works around the limitation of an HTML page being
    unable to save itself =).
    """
        log = self.get_argument("log")
            "templates/user_log.html",
            log=log,
        )

class TidyThread(threading.Thread):
    """
    Kills a user's termio session if the client hasn't updated the keepalive
    within *TIMEOUT* (global).  Also, tidies up sessions, logs, and whatnot based
    on Gate One's settings (when the time is right).

    NOTE: This is necessary to prevent shells from running eternally in the
    background.

    *session* - 45-character string containing the user's session ID
    """
    # TODO: Get this cleaning up logs according to the user's settings
    # TODO: Add the aforementioned log cleanup settings :)
    def __init__(self, session):
        threading.Thread.__init__(
            self,
            name="TidyThread-%s" % session
        )
        self.last_keepalive = datetime.now()
        self.session = session
        self.quitting = False

    def keepalive(self, datetime_obj=None):
        """
        Resets the keepalive timer.  Typically called when the user performs a new action.

        *datetime_obj* - A datetime object that will be used to measure *TIMEOUT* against.  Will end up defaulting to datetime.now() (if None) which is what you'd want 99% of the time.
        """
        if datetime_obj:
            self.last_keepalive = datetime_obj
        else:
            self.last_keepalive = datetime.now()

    def quit(self):
        self.quitting = True

    def run(self):
        while not self.quitting:
            try:
                session = self.session
                if datetime.now() > self.last_keepalive + TIMEOUT:
                    logging.info(
                        "{session} timeout.".format(
                            session=session
                        )
                    )
                    self.quitting = True
        # This loops through all the open terminals checking if each is alive
                all_dead = True
                for term in SESSIONS[session].keys():
                    try:
                        if SESSIONS[session][term]['multiplex'].alive:
                            all_dead = False
                    except TypeError: # Ignore TidyThread object
                        pass
                if all_dead:
                # Keep this low or it will take that long for the process to end
                # when it receives a SIGTERM or Ctrl-c
                time.sleep(2)
                    "Exception encountered: {exception}".format(exception=e)
                )
                self.quitting = True
            "{session} received quit()...  "
            "Killing termio session.".format(session=self.session)
        )
        # Clean up:
        for term in SESSIONS[session].keys():
            try:
                SESSIONS[session][term]['multiplex'].die()
                SESSIONS[session][term]['multiplex'].proc_kill()
            except TypeError: # Ignore the TidyThread (i.e. ourselves)
                pass
            except KeyError: # Already killed... Great!
                pass
        del SESSIONS[session]

class Application(tornado.web.Application):
    def __init__(self, settings):
        """
        Setup our Tornado application...  Everything in *settings* will wind up
        in the Tornado settings dict so as to be accessible under self.settings.
        """
        global PLUGIN_WS_CMDS
        global PLUGIN_HOOKS
        global PLUGIN_ESC_HANDLERS
        # Base settings for our Tornado app
        tornado_settings = dict(
            cookie_secret=settings['cookie_secret'],
            static_path=os.path.join(GATEONE_DIR, "static"),
            gzip=True,
            login_url="/auth"
        )
        # Make sure all the provided settings wind up in self.settings
        for k, v in settings.items():
            tornado_settings[k] = v
        # Setup the configured authentication type
        AuthHandler = NullAuthHandler # Default
        if 'auth' in settings and settings['auth']:
            if settings['auth'] == 'kerberos' and KerberosAuthHandler:
                AuthHandler = KerberosAuthHandler
                tornado_settings['sso_realm'] = settings["sso_realm"]
                tornado_settings['sso_service'] = settings["sso_service"]
            elif settings['auth'] == 'google':
                AuthHandler = GoogleAuthHandler
            logging.info("Using %s authentication" % settings['auth'])
        else:
            logging.info("No authentication method configure. All users will be"
                         " %anonymous")
        # Setup our URL handlers
        handlers = [
            (r"/recording", RecordingHandler),
            (r"/openlog", OpenLogHandler),
            (r"/docs/(.*)", tornado.web.StaticFileHandler, {
                "path": GATEONE_DIR + '/docs/build/html/',
                "default_filename": "index.html"
            })
        ]
        # Load plugins and grab their hooks
        imported = load_plugins(PLUGINS['py'])
        for plugin in imported:
            try:
                PLUGIN_HOOKS.update({plugin.__name__: plugin.hooks})
            except AttributeError:
                pass # No hooks--probably just a supporting .py file.
        # Connect the hooks
        for plugin_name, hooks in PLUGIN_HOOKS.items():
            if 'Web' in hooks:
                # Apply the plugin's Web handlers
                handlers.extend(hooks['Web'])
            if 'WebSocket' in hooks:
                # Apply the plugin's WebSocket commands
                PLUGIN_WS_CMDS.update(hooks['WebSocket'])
            if 'Escape' in hooks:
                # Apply the plugin's Escape handler
                PLUGIN_ESC_HANDLERS.update({plugin_name: hooks['Escape']})
        # This removes duplicate handlers for the same regex, allowing plugins
        # to override defaults:
        handlers = merge_handlers(handlers)
        # Include JS-only and CSS-only plugins (for logging purposes)
        js_plugins = [a.split('/')[2] for a in PLUGINS['js']]
        css_plugins = [a.split('/')[2] for a in PLUGINS['css']]
        plugin_list = list(set(PLUGINS['py'] + js_plugins + css_plugins))
        plugin_list.sort() # So there's consistent ordering
        logging.info("Loaded plugins: %s" % ", ".join(plugin_list))
        tornado.web.Application.__init__(self, handlers, **tornado_settings)

def main():
    # Simplify the auth option help message
    auths = "none, google"
    if KerberosAuthHandler:
        auths += ", kerberos"
    # Simplify the syslog_facility option help message
    facilities = FACILITIES.keys()
    facilities.sort()
    define(
        "debug",
        help="Enable debugging features such as auto-restarting when files are "
             "modified."
    )
    define("cookie_secret", # 45 chars is, "Good enough for me" (cookie joke =)
        default=None,
        help="Use the given 45-character string for cookie encryption.",
        type=str
    )
    define("command",
        default=GATEONE_DIR + "plugins/ssh/scripts/ssh_connect.py",
        help="Run the given command when a user connects (e.g. 'nethack').",
        type=str
    )
    define("address",
        default="0.0.0.0",
        help="Run on the given address.",
        type=str)
    define("port", default=443, help="Run on the given port.", type=int)
    # Please only use this if Gate One is running behind something with SSL:
    define(
        "disable_ssl",
        help="If enabled, Gate One will run without SSL (generally not a good "
             "idea)."
    )
    define(
        "certificate",
        default="certificate.pem",
        help="Path to the SSL certificate.  Will be auto-generated if none is"
             " provided.",
        type=str
    )
    define(
        "keyfile",
        default="keyfile.pem",
        help="Path to the SSL keyfile.  Will be auto-generated if none is"
        type=str
    )
    define(
        "user_dir",
        default=GATEONE_DIR + "/users",
        help="Path to the location where user files will be stored.",
        type=str
    )
    define(
        "session_dir",
        default="/tmp/gateone",
        help="Path to the location where session information will be stored.",
        type=str
    )
    define(
        "session_logging",
        help="If enabled, logs of user sessions will be saved in "
             "<user_dir>/logs.  Default: Enabled"
    )
    define( # This is an easy way to support cetralized logging
        "syslog_session_logging",
        help="If enabled, logs of user sessions will be written to syslog."
    )
    define(
        "syslog_facility",
        default="daemon",
        help="Syslog facility to use when logging to syslog (if "
             "syslog_session_logging is enabled).  Must be one of: %s."
             "  Default: daemon" % ", ".join(facilities),
        type=str
    )
    define(
        "session_timeout",
        default="5d",
        help="Amount of time that a session should be kept alive after the "
        "client has logged out.  Accepts <num>X where X could be one of s, m, h"
        ", or d for seconds, minutes, hours, and days.  Default is '5d' (5 days"
        ").",
        type=str
    )
    define(
        "auth",
        help="Authentication method to use.  Valid options are: %s" % auths,
        type=str
    )
    define(
        "sso_realm",
        help="Kerberos REALM (aka DOMAIN) to use when authenticating clients."
             " Only relevant if Kerberos authentication is enabled.",
        type=str
    )
    define(
        "sso_service",
        default='HTTP',
        help="Kerberos service (aka application) to use. Defaults to HTTP. "
             "Only relevant if Kerberos authentication is enabled.",
        type=str
    )
    define(
        "embedded",
        help="Run Gate One in Embedded Mode (no toolbar, only one terminal "
             "allowed, etc.  See docs)."
    )
    define(
        "dtach",
        help="Wrap terminals with dtach. Allows sessions to be resumed even if "
             "Gate One is stopped and started (which is a sweet feature =)."
    )
    define(
        "kill",
        help="Kill any running Gate One terminal processes including dtach'd "
             "processes."
    )
    # TODO: Give plugins the ability to add their own define()s
    # TODO: Use the arguments passed to gateone.py to generate server.conf if it
    #       isn't already present.
    if os.path.exists(GATEONE_DIR + "/server.conf"):
        tornado.options.parse_config_file(GATEONE_DIR + "/server.conf")
    else: # Generate a default server.conf with a random cookie secret
        logging.info("No server.conf found.  A new one will be generated using "
                     "defaults.")
        if not os.path.exists(options.user_dir): # Make our user_dir
            mkdir_p(options.user_dir)
            os.chmod(options.user_dir, 0700)
        if not os.path.exists(options.session_dir): # Make our session_dir
            mkdir_p(options.session_dir)
            os.chmod(options.session_dir, 0700)
        config_defaults = {
            'debug': False,
            'cookie_secret': generate_session_id(), # Works for so many things!
            'port': 443,
            'address': '0.0.0.0', # All addresses
            'embedded': False,
            'auth': None,
            'dtach': True,
            # NOTE: The next four options are specific to the Tornado framework
            'log_file_max_size': 100 * 1024 * 1024, # 100MB
            'log_file_num_backups': 10, # 1GB total max
            'log_file_prefix': '/var/log/gateone/webserver.log',
            'logging': 'info', # One of: info, warning, error, none
            'user_dir': options.user_dir,
            'session_dir': options.session_dir,
            'session_logging': options.session_logging,
            'syslog_session_logging': options.syslog_session_logging,
            'syslog_facility': options.syslog_facility,
            'session_timeout': options.session_timeout,
            'keyfile': GATEONE_DIR + "/keyfile.pem",
            'certificate': GATEONE_DIR + "/certificate.pem",
            'command': (
                GATEONE_DIR + "/plugins/ssh/scripts/ssh_connect.py -S "
                r"'/tmp/gateone/%SESSION%/%r@%h:%p' -a "
                "'-oUserKnownHostsFile=%USERDIR%/%USER%/known_hosts'"
            ),
            'sso_realm': 'EXAMPLE.COM',
            'sso_service': 'HTTP'
        }
        config = open(GATEONE_DIR + "/server.conf", "w")
        for key, value in config_defaults.items():
            if isinstance(value, basestring):
                config.write('%s = "%s"\n' % (key, value))
            else:
                config.write('%s = %s\n' % (key, value))
        config.close()
    # Create the log dir if not already present (NOTE: Assumes we're root)
    log_dir = os.path.split(options.log_file_prefix)[0]
    if not os.path.exists(log_dir):
        try:
            mkdir_p(log_dir)
        except OSError:
            print("\x1b[1;31mERROR:\x1b[0m Could not create %s for "
                  "log_file_prefix: %s" % (log_dir, options.log_file_prefix))
            print("You probably want to change this option, run Gate One as "
                  "root, or create that directory and give the proper user "
                  "ownership of it.")
            sys.exit(1)
    tornado.options.parse_command_line()
    if options.kill:
        # Kill all running dtach sessions (associated with Gate One anyway)
        killall(options.session_dir)
        sys.exit(0)
    # Set our CMD variable to tell the multiplexer which command to execute
    global CMD
    CMD = options.command
    # Set our global session timeout
    global TIMEOUT
    TIMEOUT = convert_to_timedelta(options.session_timeout)
    # Define our Application settings
    app_settings = {
        'gateone_dir': GATEONE_DIR, # Only here so plugins can reference it
        'debug': options.debug,
        'cookie_secret': options.cookie_secret,
        'auth': none_fix(options.auth),
        'embedded': str2bool(options.embedded),
        'user_dir': options.user_dir,
        'session_dir': options.session_dir,
        'session_logging': options.session_logging,
        'syslog_session_logging': options.syslog_session_logging,
        'syslog_facility': options.syslog_facility,
        'dtach': options.dtach,
        'sso_realm': options.sso_realm,
        'sso_service': options.sso_service
    }
    # Check to make sure we have a certificate and keyfile and generate fresh
    # ones if not.
    if not os.path.exists(options.keyfile):
        logging.info("No SSL private key found.  One will be generated.")
        gen_self_signed_ssl()
    if not os.path.exists(options.certificate):
        logging.info("No SSL certificate found.  One will be generated.")
    logging.info(
        "Listening on https://{address}:{port}/".format(
            address=options.address, port=options.port
        )
    )
    # Setup static file links for plugins (if any)
    static_dir = os.path.join(GATEONE_DIR, "static")
    plugin_dir = os.path.join(GATEONE_DIR, "plugins")
    create_plugin_static_links(static_dir, plugin_dir)
    # Instantiate our Tornado web server
    ssl_options = {
        "certfile": os.path.join(os.getcwd(), "certificate.pem"),
        "keyfile": os.path.join(os.getcwd(), "keyfile.pem"),
    }
    if options.disable_ssl:
        ssl_options = None
    http_server = tornado.httpserver.HTTPServer(
        Application(settings=app_settings), ssl_options=ssl_options)
    try: # Start your engines!
        http_server.listen(options.port, options.address)
        tornado.ioloop.IOLoop.instance().start()
    except KeyboardInterrupt: # ctrl-c
        logging.info("Caught KeyboardInterrupt.  Killing sessions...")
        for t in threading.enumerate():
            if t.getName().startswith('TidyThread'):
                t.quit()

    main()

#
#

# Meta

About The SSO Module
====================
sso.py is a Tornado Single Sign-On (SSO) authentication module that implements
GSSAPI authentication via python-kerberos (import kerberos).  If "Negotiate"
authentication (GSSAPI SSO) fails it will gracefully fall back to "Basic" auth
(authenticating a given username/password against your Kerberos realm).

For this module to work you must add 'sso_realm' and 'sso_service' to your
Tornado application's settings.  See the docstring of the KerberosAuthMixin for
how to do this.

This module should work with regular MIT Kerberos implementations as well as
Active Directory (Heimdal is untested but should work fine).  If you're
experiencing trouble it is recommended that you set debug=True in your
application settings.  This will enable printing of Kerberos exception messages.

Troubleshooting:

If your browser asks you for a password (i.e. SSO failed) there's probably
something wrong with your Kerberos configuration on either the client or the
server (usually it's a problem with forward/reverse DNS resolution or an
incorrect or missing service principal in your keytab).

If you're using Active Directory, make sure that there's an HTTP
servicePrincipalName (SPN) matching the FQDN of the host running your Tornado
server.  For example:  HTTP/somehost.somedomain.com@CORP.MYCOMPANY.COM
You may also want a short hostname SPN: HTTP/somehost@CORP.MYCOMPANY.COM

Also make sure that the service principal is in upper case as most clients (
web browsers) will auto-capitalize the principal when verifying the server.

Here's some things to test in order to find problems with your Kerberos config:

Try these from both the client and the server (NOTE: Assuming both are Unix):
kinit -p <user@REALM> # To verify you can authenticate via Kerberos (at all)
nslookup <server FQDN> # To verify the IP address reverse maps properly (below)
nslookup <IP address that 'server FQDN' resolves to>
kvno HTTP/somehost.somedomain.com # To verify your service principal

Remember: Kerberos is heavily dependent on DNS to verify the server and client
are who they claim to be.

I find that it is useful to get GSSAPI authentication working with OpenSSH first
before I attempt to get a custom service principal working with other
applications.  This is because SSH uses the HOST/ prinicipal which is often
taken care of automatically via most Kerberos management tools (including AD).
If you can get SSO working with SSH you can get SSO working with anything else.

"""

import httplib, logging

# 3rd party modules
import kerberos
from tornado.escape import utf8
from tornado.util import b

# NOTE: For some reason if I set this as just an 'object' it doesn't work.
class KerberosAuthMixin(tornado.web.RequestHandler):
    """
    Authenticates users via Kerberos-based Single Sign-On.  Requires that you
    define 'sso_realm' and 'sso_service' in your Tornado Application settings.
    For example::

        settings = dict(
            cookie_secret="iYR123qg4UUdsgf4CRung6BFUBhizAciid8oq1YfJR3gN",
            static_path=os.path.join(os.path.dirname(__file__), "static"),
            login_url="/auth",
            debug=True,
            sso_realm="EXAMPLE.COM",
            sso_service="HTTP" # Should pretty much always be HTTP
        )

    NOTE: If you're using 'HTTP' as the service it must be in all caps or it
    might not work with some browsers/clients (which auto-capitalize all
    services).

    To implement this mixin::

        from sso import KerberosAuthMixin
        class KerberosAuthHandler(tornado.web.RequestHandler, KerberosAuthMixin):

            def get(self):
                auth_header = self.request.headers.get('Authorization')
                if auth_header:
                    self.get_authenticated_user(self._on_auth)
                    return
                self.authenticate_redirect()

            def _on_auth(self, user):
                if not user:
                    raise tornado.web.HTTPError(500, "Kerberos auth failed")
                self.set_secure_cookie("user", tornado.escape.json_encode(user))
                print("KerberosAuthHandler user: %s" % user) # To see what you get
                next_url = self.get_argument("next", None) # To redirect properly
                if next_url:
                    self.redirect(next_url)
                else:
                    self.redirect("/")
    """
    def initialize(self):
        """
        Print out helpful error messages if the requisite settings aren't
        configured.

        NOTE: It won't hurt anything to override this method in your
        RequestHandler.
        """
        self.require_setting("sso_realm", "Kerberos/GSSAPI Single Sign-On")
        self.require_setting("sso_service", "Kerberos/GSSAPI Single Sign-On")

    def get_authenticated_user(self, callback):
        """
        Processes the client's Authorization header and calls
        self.auth_negotiate() or self.auth_basic() depending on what headers
        were provided by the client.
        """
        auth_header = self.request.headers.get('Authorization')
        if auth_header.startswith('Negotiate'):
            self.auth_negotiate(auth_header, callback)
        elif auth_header.startswith('Basic '):
            self.auth_basic(auth_header, callback)

    def auth_negotiate(self, auth_header, callback):
        """
        Perform Negotiate (GSSAPI/SSO) authentication via Kerberos.
        """
        auth_str = auth_header.split()[1]
        # Initialize Kerberos Context
        result, context = kerberos.authGSSServerInit(
            self.settings['sso_service'])
        if result != 1:
            raise tornado.web.HTTPError(500, "Kerberos Init failed")
        result = kerberos.authGSSServerStep(context, auth_str)
        if result == 1:
            gssstring = kerberos.authGSSServerResponse(context)
        else: # Fall back to Basic auth
        # NOTE: The user we get from Negotiate is a full UPN (e.g. user@REALM)
        user = kerberos.authGSSServerUserName(context)
        self.set_header('WWW-Authenticate', "Negotiate %s" % gssstring)
        kerberos.authGSSServerClean(context)
        callback(user)

    def auth_basic(self, auth_header, callback):
        """
        Perform Basic authentication using Kerberos against
        self.settings['sso_realm'].
        """
        auth_decoded = base64.decodestring(auth_header[6:])
        username, password = auth_decoded.split(':', 2)
        try:
            kerberos.checkPassword(
                username,
                password,
                self.settings['sso_service'],
                self.settings['sso_realm'])
        except Exception as e: # Basic auth failed
            if self.settings['debug']:
                print(e) # Very useful for debugging Kerberos errors
            return self.authenticate_redirect()
        # NOTE: Basic auth just gives us the username without the @REALM part
        #       so we have to add it:
        user = "%s@%s" % (username, self.settings['sso_realm'])

    def authenticate_redirect(self):
        """
        Informs the browser that this resource requires authentication (status
        code 401) which should prompt the browser to reply with credentials.

        The browser will be informed that we support both Negotiate (GSSAPI/SSO)
        and Basic auth.

        NOTE: I know this isn't technically a redirect but I wanted to make this
        process as close as possible to how things work in tornado.auth.
        """
        if self._headers_written:
            raise Exception('Headers have already been written')
        self.set_status(401)
        self.add_header("WWW-Authenticate", "Negotiate")
        self.add_header(
            "WWW-Authenticate",
            'Basic realm="%s"' % self.settings['sso_realm']
        )
        self.finish()

#
#

# TODO: See if we can spin off termio.py into its own little program that sits between Gate One and ssh_connect.py.  That way we can take advantage of multiple cores/processors (for terminal-to-HTML processing).  There's no reason why we can't write something that does what dtach does.  Just need to redirect the fd of self.cmd to a unix domain socket and os.setsid() somewhere after forking (twice maybe?).
# TODO: Make the environment variables used before launching self.cmd configurable

# Meta

About termio
============
This module provides a Multiplex class that can perform the following:

 * Fork a child process that opens a given terminal program.
 * Read and write data to and from the child process.
 * Log the output of the child process to a file and/or syslog.

The Multiplex class is meant to be used in conjunction with a running Tornado
IOLoop instance.  It can be instantiated from within your Tornado application
like so::

    multiplexer = termio.Multiplex(
        'nethack',
        tmpdir='/tmp',
        log_path='/var/log/myapp',
        user='bsmith@CORP',
        term_num=1,
        syslog=True
    )

Then *multiplexer* can create and launch a new controlling terminal (tty)
running the given command (e.g. 'nethack')::

    env = {
        'PATH': os.environ['PATH'],
        'MYVAR': 'foo'
    }
    fd = multiplexer.create(80, 24, env=env)
    # The fd is returned from create() in case you want more low-level control.

Input and output from the controlled program is asynchronous and gets handled
via IOLoop.  It will automatically write all output from the terminal program to
an instance of self.terminal_emulator (which defaults to Gate One's
terminal.Terminal).  So if you want to perform an action whenever the running
terminal application has output (like, say, sending a message to a client)
you'll need to attach a callback::

    def screen_update():
        'Called when new output is ready to send to the client'
        output = multiplexer.dumplines()
        socket_or_something.write(output)
    multiplexer.callbacks[multiplexer.CALLBACK_UPDATE] = screen_update

In this example, screen_update() will write() the output of
multiplexer.dumplines() to *socket_or_something* whenever the terminal program
has some sort of output.  You can also make calls directly to the terminal
emulator (if you're using a custom one)::

        output = multiplexer.term.my_custom_func()
        whatever.write(output)

Writing characters to the controlled terminal application is pretty

    multiplexer.proc_write('some text')

Typically you'd pass in keystrokes or commands from your application to the
underlying program this way and the screen/terminal emulator would get updated
automatically.  If using Gate One's Terminal() you can also attach callbacks
to perform further actions when more specific situations are encountered (e.g.
when the window title is set via that respective escape sequence)::

    def set_title():
        'Hypothetical title-setting function'
        print("Window title was just set to: %s" % multiplexer.term.title)
    multiplexer.term.callbacks[multiplexer.CALLBACK_TITLE] = set_title

Module Functions and Classes
============================
"""

# Stdlib imports
import signal, threading, fcntl, os, pty, re, sys, time, termios, struct
import io, codecs, gzip, syslog
from tornado import ioloop
from itertools import izip
from subprocess import Popen

from utils import noop

# Globals
# NOTE: That unicode character was carefully selected from only the finest
# of the PUA.  I hereby dub thee, "U+F0F0F0, The Separator."

# Helper functions
def handle_special(e):
    """
    Used in conjunction with codecs.register_error, will replace special ascii
    characters such as 0xDA and 0xc4 (which are used by ncurses) with their
    Unicode equivalents.
    """
    # TODO: Fill this out with *all* the ascii characters >127 from
    #       http://www.ascii-code.com/
    specials = {
        # NOTE: When $TERM is set to "Linux" these end up getting used by things
        #       like ncurses-based apps.  In other words, it makes a whole lot
        #       of ugly look pretty again.
        0xda: u'┌', # ACS_ULCORNER
        0xc0: u'└', # ACS_LLCORNER
        0xbf: u'┐', # ACS_URCORNER
        0xd9: u'┘', # ACS_LRCORNER
        0xb4: u'├', # ACS_RTEE
        0xc3: u'┤', # ACS_LTEE
        0xc1: u'┴', # ACS_BTEE
        0xc2: u'┬', # ACS_TTEE
        0xc4: u'─', # ACS_HLINE
        0xb3: u'│', # ACS_VLINE
        0xc5: u'┼', # ACS_PLUS
        0x2d: u'', # ACS_S1
        0x5f: u'', # ACS_S9
        0xc5: u'◆', # ACS_DIAMOND
        0xb2: u'▒', # ACS_CKBOARD
        0xf8: u'°', # ACS_DEGREE
        0xf1: u'±', # ACS_PLMINUS
        0xf9: u'•', # ACS_BULLET
        0x3c: u'←', # ACS_LARROW
        0x3e: u'→', # ACS_RARROW
        0x76: u'↓', # ACS_DARROW
        0x5e: u'↑', # ACS_UARROW
        0xb0: u'⊞', # ACS_BOARD
        0x0f: u'⨂', # ACS_LANTERN
        0xdb: u'█', # ACS_BLOCK
        0x9d: u'Ø', # Upper-case slashed zero (157)--using same as empty set
        0xd8: u'Ø', # Empty set (216)
# Note to self:  Why did I bother with these overly descriptive comments?  Ugh
# I've been staring at obscure symbols far too much lately ⨀_⨀
        0xc7: u'Ç', # Latin capital letter C with cedilla
        0xeb: u'ë', # Latin small letter e with diaeresis
        0x99: u'™', # Trademark sign--in case you didn't know!
        0xff: u'ÿ', # Latin small letter y with diaeresis⬅So THATS what that is
        0xa8: u'¨', # Spacing diaeresis - umlaut
        0xec: u'ì', # Latin small letter i with grave... concern.
        0xca: u'Ê', # Latin capital letter E with circumflex
        0x83: u'ƒ', # Latin small letter f with hook
    }
    # I left this in its odd state so I could differentiate between the two
    # in the future.
    if isinstance(e, (UnicodeEncodeError, UnicodeTranslateError)):
        s = [u'%s' % specials[ord(c)] for c in e.object[e.start:e.end]]
        return ''.join(s), e.end
    else:
codecs.register_error('handle_special', handle_special)

# Classes
class Multiplex:
    """
    The Multiplex class takes care of forking a child process and provides
    methods for reading/writing to it.  It also creates an instance of
    tornado.ioloop.IOLoop that listens for events on the spawned terminal
    application and updates self.proc[fd]['term'] with any changes.
    """
    CALLBACK_UPDATE = 1 # Screen update
    CALLBACK_EXIT = 2   # When the underlying program exits

    def __init__(self,
            cmd=None,
            terminal_emulator=None, # Defaults to Gate One's terminal.Terminal
            cps=5000,
            tmpdir="/tmp",
            log_path=None,
            user=None, # Only used by syslog output (to differentiate who's who)
            term_num=None, # Also only for syslog output for the same reason
            syslog=False,
            syslog_facility=syslog.LOG_DAEMON):
        # NOTE: Commented this out because Death apparently moves too swiftly!
        # Elect for automatic child reaping (may Death take them kindly!)
        #signal.signal(signal.SIGCHLD, signal.SIG_IGN)
        self.cmd = cmd
        if not terminal_emulator:
            # Why do this?  So you could use/write your own specialty emulator.
            # Whatever you use it just has to accept 'rows' and 'cols' as
            # keyword arguments in __init__()
            from terminal import Terminal # Dynamic import to cut down on waste
            self.terminal_emulator = Terminal
        else:
            self.terminal_emulator = terminal_emulator
        self.cps = cps # Characters per second where the rate limiter kicks in
        self.tmpdir = tmpdir # Where to store things like pid files
        self.log_path = log_path # Logs of the terminal output wind up here
        self.syslog = syslog # See "if self.syslog:" below
        self.syslog_facility = syslog_facility # Ditto
        self.lock = threading.RLock() # Used by the @synchronized decorator
        self.io_loop = ioloop.IOLoop.instance() # Monitors child for activity
        self.alive = True # Provides a quick way to see if we're still kickin'
        # These three variables are used by the rate limiting function:
        self.ratelimit = time.time()
        self.skip = False
        self.ratelimiter_engaged = False
        # Setup our callbacks
        self.callbacks = { # Defaults do nothing which saves some conditionals
            self.CALLBACK_UPDATE: noop,
            self.CALLBACK_EXIT: noop,
        }
        # Configure syslog logging
        self.user = user
        self.term_num = term_num
        self.syslog_buffer = ''
        if self.syslog: # Dynamic imports again because I'm freaky and frugal
            import syslog
            # Sets up syslog messages to show up like this:
            #   Sep 28 19:45:02 <hostname> gateone: <log message>
            syslog.openlog('gateone', 0, syslog_facility)

    def create(self, rows=24, cols=80, env=None):
        """
        Creates a new virtual terminal (tty) and executes self.cmd within it.
        Also sets up our read/write callback and attaches them to Tornado's
        IOLoop.

        *cols*
            The number of columns to emulate on the virtual terminal (width)
        *rows*
            The number of rows to emulate (height).
        *env*
            A dictionary of environment variables to set when executing self.cmd.
        """
        pid, fd = pty.fork()
        if pid == 0: # We're inside the child process
            try: # Enumerate our file descriptors
                fd_list = [int(i) for i in os.listdir('/proc/self/fd')]
                fd_list = xrange(256)
    # Close all file descriptors other than stdin, stdout, and stderr (0, 1, 2)
            for i in [i for i in fd_list if i > 2]:
                try:
                    os.close(i)
                    pass
            if not env:
                env = {}
            env["COLUMNS"] = str(cols)
            env["LINES"] = str(rows)
            env["TERM"] = "xterm" # TODO: This needs to be configurable on-the-fly
            #env["PATH"] = os.environ['PATH']
            #env["LANG"] = os.environ['LANG']
            p = Popen(self.cmd, env=env, shell=True)
            p.wait()
            # This exit() ensures IOLoop doesn't freak out about a missing fd
            sys.exit(0)
        else: # We're inside this Python script
            fcntl.fcntl(fd, fcntl.F_SETFL, os.O_NONBLOCK)
            # These two lines set the size of the terminal window:
            s = struct.pack("HHHH", rows, cols, 0, 0)
            fcntl.ioctl(fd, termios.TIOCSWINSZ, s)
            self.fd = fd
            self.pid = pid
            self.term = self.terminal_emulator(rows=rows, cols=cols)
            self.time = time.time()
            # NOTE: io.open has a memory leak in Python 2.6 and below so we only
            #       use it for Python 2.7+.  Why use it at all?  It's faster,
            #       more memory efficient, and not deprecated.
            if sys.version_info[0] == 2 and sys.version_info[1] >= 7:
                self.reader =  io.open(
                    self.fd,
                    'rt',
                    buffering=1024,
                    newline="",
                    encoding='UTF-8', # TODO: Make this configurable
                    closefd=False,
                    errors='handle_special'
                )
            else:
            # For Python <= 2.6
                self.reader = os.fdopen(self.fd, 'r')
            # Tell our IOLoop instance to start watching the child
            self.io_loop.add_handler(
                self.fd, self.proc_read, self.io_loop.READ)
            self.prev_output = [None for a in xrange(rows-1)]
            return fd

    def die(self):
        """
        Sets self.alive to False

        NOTE: This is actually important as it allows controlling processes to
        see if the multiplexer is still alive or not (so they don't have to
        enumerate the process table looking for a particular pid).
        """
        self.alive = False

        """
        Resizes the child process's terminal window to *rows* and *cols*
        """
        self.term.resize(rows, cols)
        s = struct.pack("HHHH", rows, cols, 0, 0)
        fcntl.ioctl(self.fd, termios.TIOCSWINSZ, s)

    def redraw(self):
        """
        Tells the running terminal program to redraw the screen by executing
        a window resize event (using its current dimensions) and writing a
        ctrl-l.
        """
        s = struct.pack("HHHH", self.term.rows, self.term.cols, 0, 0)
        self.proc_write(u'\x0c') # ctrl-l
        # WINCH can mess things up quite a bit so I've disabled it.
        # Leaving this here in case it ever becomes configurable (reference)
        #os.kill(self.proc[fd]['pid'], signal.SIGWINCH)

    def proc_kill(self):
        """
        Kill the child process associated with the given file descriptor (fd).

        NOTE: If dtach is being used this only kills the dtach process.
        """
        try:
            self.io_loop.remove_handler(self.fd)
            print("Error trying to remove fd: %s" % self.fd)
        try:
            os.kill(self.pid, signal.SIGTERM)
            os.wait()
        except (IOError, OSError):
            # Lots of trivial reasons why we could get these
            pass

    def term_write(self, chars):
        """
        Writes *chars* to self.term and also takes care of logging to
        *self.log_path* (if set) and/or syslog (if *self.syslog* is True).

        NOTE: This kind of logging doesn't capture user keystrokes.  This is
        intentional as we don't want passwords winding up in the logs.
        """
        self.term.write(chars)
        # Write to the log too (if configured)
        if self.log_path:
            # NOTE: I'm using an obscure unicode symbol in order to avoid
            # conflicts.  We need to do our best to ensure that we can
            # differentiate between terminal output and our log format...
            # This should do the trick because it is highly unlikely that
            # someone would be displaying this obscure unicode symbol on an
            # actual terminal unless they were using Gate One to view a
            # Gate One log file in vim or something =)
            # \U000f0f0f == U+F0F0F (Private Use Symbol)
            output = u"%s:%s\U000f0f0f" % (now, chars)
            log = gzip.open(self.log_path, mode='a')
            log.write(output.encode("utf-8"))
            log.close()
        # NOTE: Gate One's log format is special in that it can be used for both
        # playing back recorded sessions *or* generating syslog-like output.
        if self.syslog:
            # Try and keep it as line-line as possible so we don't end up with
            # a log line per character.
            if '\n' in chars:
                for line in chars.splitlines():
                    if self.syslog_buffer:
                        line = self.syslog_buffer + line
                        self.syslog_buffer = ''
                    syslog.syslog("%s %s: %s" % (
                        self.user, self.term_num, line))
            else:
                self.syslog_buffer += chars

    def proc_read(self, fd, event):
        """
        Read in the output of the process associated with *fd* and write it to
        self.term.

        This method will also keep an eye on the output rate of the underlying
        terminal application.  If it goes to high (which would gobble up CPU) it
        will engage a rate limiter.  So if someone thinks it would be funny to
        run 'top' with a refresh rate of 0.01 they'll really only be getting
        updates every ~2 seconds (and it won't bog down the server =).

        NOTE: This method is not meant to be called directly...  The IOLoop
        should be the one calling it when it detects an io_loop.READ event.
        """
        self.lock.acquire()
        if event == self.io_loop.READ:
            try:
                updated = self.reader.read(65536)
                #updated = os.read(fd, 65536) # A lot slower than reader, why?
                ratelimit = self.ratelimit
                now = time.time()
                timediff = now - self.time
                rate_timediff = now - ratelimit
                characters = len(updated)
                cps = characters/timediff # Assumes 7 bits per char (ASCII)
                # The conditionals below drop the output of our fd if it's coming
                # in too fast.  Essentially, it is a rate limiter to prevent
                # really noisy/fast output console apps (say, 'top' with a
                # refresh rate of 0.01) from causing this application to
                # gobble up all the system CPU trying to process the input.
                # Think of it like mplayer's "-framedrop" option that keeps
                # your video playing at the proper rate if the CPU runs out of
                # power to process video frames.

                # Only consider dropping if the rate is faster than self.cps:
                if cps > self.cps:
                    # Don't start cutting frames unless this is a constant thing
                    if rate_timediff > 5:
                        # TODO: Have this flash a message on the screen
                        #       indicating the rate limiter has been engaged.
                        self.ratelimiter_engaged = True
                        check = divmod(now - ratelimit, 2)[0]
                        # Update once every other second or so
                        if check % 2 == 0 and not self.skip:
                            self.term_write(updated)
                            self.skip = True
                            self.callbacks[self.CALLBACK_UPDATE]()
                        elif self.skip:
                            self.skip = False
                    else:
                        self.term_write(updated)
                        self.callbacks[self.CALLBACK_UPDATE]()
                    # NOTE: This can result in odd output with too-fast apps
                else:
                    self.term_write(updated)
                if now - ratelimit > 1:
                    # Reset the rate limiter
                    self.ratelimit = time.time()
                self.callbacks[self.CALLBACK_UPDATE]()
                self.time = time.time()
            except KeyError as e:
                # Should just be an exception from handle_special()
                logging.debug("KeyError in proc_read(): %s" % e) # So we know
            except (IOError, OSError) as e:
                logging.error("Got exception in proc_read: %s" % `e`)
                self.die()
                self.proc_kill()
                import traceback
                logging.error("Got BIZARRO exception in proc_read (WTF?): %s" % `e`)
                traceback.print_exc(file=sys.stdout)
        else: # Child died
            logging.debug("Apparently fd %s just died." % self.fd)
            self.die()
            self.proc_kill()
            self.callbacks[self.CALLBACK_EXIT]()
        self.lock.release()

    def proc_write(self, chars):
        """
        Writes *chars* to the terminal process running on *fd*.
        """
        try:
            # By creating a new writer with every execution of this function we
            # can avoid the memory leak in earlier versions of Python.  It is
            # slightly slower but, hey, now you have an excuse to upgrade!
            writer = io.open(
                self.fd,
                'wt',
                buffering=1024,
                newline="",
                encoding='UTF-8',
                closefd=False
            )
            writer.write(chars)
            writer.flush()
            #os.write(self.fd, s) # This doesn't leak but it also doesn't support unicode
        except Exception as e:
            logging.error("proc_write() exception: %s" % e)

    def dumplines(self):
        """
        Returns the terminal text lines (a list of lines, to be specific) and
        its scrollback buffer (which is also a list of lines) as a tuple,
        (scrollback, text).  If a line hasn't changed since the last dump then
        it will be replaced with an empty string (in the terminal text lines).
        """
        try:
            output = []
            scrollback, html = self.term.dump_html()
            for line1, line2 in izip(self.prev_output, html):
                if line1 != line2:
                    output.append(line2)
                else:
                    output.append('')
            self.prev_output = html
            return (scrollback, output)
            return (None, None)

command line option; --session_timeout=string (special: [0-9]+[smhd])

command line option; --sso_realm=string (Kerberos realm or Active Directory domain)

command line option; --sso_service=string (kerberos service name)

command line option; --syslog_facility=string (auth|cron|daemon|kern|local0|local1|local2|local3|local4|local5|local6|local7|lpr|mail|news|syslog|user|uucp)

GateOne.terminals[num].backspace <character> (GateOne.terminals[num] attribute)

GateOne.terminals[num].columns <number> (GateOne.terminals[num] attribute)

GateOne.terminals[num].created <Date()> (GateOne.terminals[num] attribute)

GateOne.terminals[num].mode <string> (GateOne.terminals[num] attribute)

GateOne.terminals[num].playbackFrames <Array()> (GateOne.terminals[num] attribute)

GateOne.terminals[num].prevScreen <Array()> (GateOne.terminals[num] attribute)

GateOne.terminals[num].rows <number> (GateOne.terminals[num] attribute)

GateOne.terminals[num].screen <Array()> (GateOne.terminals[num] attribute)

GateOne.terminals[num].scrollback <Array()> (GateOne.terminals[num] attribute)

GateOne.terminals[num].scrollbackVisible <boolean> (GateOne.terminals[num] attribute)

GateOne.terminals[num].sshConnectString <string> (GateOne.terminals[num] attribute)

GateOne.Terminal.closeTermCallbacks (GateOne.Terminal attribute)

GateOne.Terminal.reattachTerminalsAction() (GateOne.Terminal method)

GateOne.Terminal.reconnectTerminalAction() (GateOne.Terminal method)

GateOne.Terminal.updateTerminalAction() (GateOne.Terminal method)