December 2012 Archives

Week 14: Readings

| 0 Comments | 0 TrackBacks

This week, there are 3 papers to read and prepare to discuss. One has no visual analytics, but has the potential for visual analytics to enhance the work. The other are primarily visual analytics papers, with the last being a view of the agenda in 2006 that is worth reflecting on here in 2012.

  • Henry E. Brady and John E. McNulty, 2011. Turning Out to Vote: The Costs of Finding and Getting to the Polling Place. American Political Science Review 105, 115-134. {This paper has no visual analytics methods applies. Your focus in reading it should be on how visual analytics could be applied and what kinds.} Suggested by Molly.
  • Correa, C.D.; Yu-Hsuan Chan; Kwan-Liu Ma; , "A framework for uncertainty-aware visual analytics," Visual Analytics Science and Technology, 2009. VAST 2009. IEEE Symposium on , vol., no., pp.51-58, 12-13 Oct. 2009. {This short paper introduces a visual analytics framework for dealing with uncertainty in large, numerical data sets. The case study on housing data is comparable in some ways to the data in the Brady and McNulty paper on polling places. Thus, one thing to consider is potential applicability of the approach outlined to analyses such as theirs.}   Suggested by Dong.
  • Keim, D.A., Mansmann, F., Schneidewind, J. and Ziegler, H. 2006: Challenges in Visual Data Analysis. Information Visualization, 2006, 9-16. {This short paper outlined a research agenda for visual analytics back in 2006. As the last paper we read collectively for the term, we will use it to reflect on everything you have read and what you all see as challenges that remain from those outlined or new ones that were not articulated then.} Suggested by Muhammed.

Week 13 Readings & Media

| 0 Comments | 0 TrackBacks

This week, there are 2 papers to read and prepare to discuss and one video. The topics are very different, but they have space-time analysis in common.

  • Kwan, M. (2002a). Feminist visualization: re-envisioning GIS as a method in feminist geographic research. Annals of the Association of American Geographers. 92(4): 645-661. {This paper is a representative example of the potential to bridge between some research questions in human geography and those in geovisualization (and visual analytics). Specifically, Kwan explores ways in which GIS (and 3D visualization) can be used to address questions in feminist geographic research.} Suggested by Jenna.
  • Fabrikant, S. I., Rebich-Hespanha, S., Andrienko, N., Andrienko, G., & Montello, D. R. (2008). Novel method to measure inference affordance in static small-multiple map displays representing dynamic processes. The Cartographic Journal, 45(3), 201-215. {This paper offers an interesting mix between a cognitively-oriented strategy for studying how (and how well) different visual display forms work and application of visual analytics to that study. Thus, it builds on the paper Lam, et al paper on evaluation scenarios while also introducing some visual analytics strategies to analyze movement data - eye movements in this case.} Suggested by Jenny.
  • Jo Wood, et al, 2011. giCentre VAST Contest 2011 - Mini Challenge 1 - http://vimeo.com/30781990 Read the 1/2 page intro to Mini Challenge 1: http://hcil.cs.umd.edu/localphp/hcil/vast11/index.php/taskdesc/ {There is no paper to read with this, but a ½ page intro to a challenge task and a video detailing results. The data available was a hypothetical set of twitter data and the challenge was about a disease outbreak. In watching the video, give particular attention to the visual/statistical methods}

Cleaning up your account

| 0 Comments | 0 TrackBacks
For the first year after you sign up for Amazon Web Services, you have enough free usage to run a small server for a year. In addition, you should have enough credit to do some larger computational usage. However, after a year you will start to be billed for your use of AWS resources. So, you should either cancel your account, or release resources you are not using.

If you are done with AWS for the foreseeable future, then you should cancel your account. To do this, go to this page: https://portal.aws.amazon.com/gp/aws/manageYourAccount

Alternatively, if you want to keep your account, it is wise to release the resources that you are using, so that you are not billed for them. The most important principle to avoid surprises is to examine your account. Go to this URL: http://aws.amazon.com/account/, and then click on Account Activity. 

Next, scroll down to the "Details" section, and click on the "plus" symbol next to "Amazon Elastic Compute Cloud". This should show you your usage. 

Next, log back into the AWS console and go the the EC account section, and delete the following resources: Instances, Volumes, and Elastic IPs. This should account for the resources you were encouraged to use in this course. Also, put a reminder in your calendar to delete your account when one year is up. 

After you've given your instance about 20 minutes to configure Windows, you can get ready to log in to the instance and start working with your software.

  1. Open Windows Remote Desktop. In most versions of Windows, you can browse to this from Start > All Programs > Accessories > Remote Desktop Connection. In older versions of Windows, it may be in a folder called Communications. 

    Remote Desktop is a program that you can use to log in to other computers from your own computer. If you're new to Remote Desktop, you may want to take some time to read Remote Desktop Connection: frequently asked questions.
  2. In Remote Desktop Connection, click the Options button > Local Resources tab > More button and check the box forDrives. Then click OK. This will permit you to copy data from your machine on to the remote machine (in this case, your Amazon EC2 instance).
  3. In Remote Desktop Connection, under the General tab, type or paste the Elastic IP of your instance into the Computerinput box. If you can't remember what this is, click Elastic IPs in the AWS Management Console and you will see it listed.
  4. In the User name input box, type Administrator. Then click the Connect button.

    You might see a warning message here about remote desktop connections harming your computer. Any time you connect to a remote computer, there is the possibility that a malicious party could try to pose as the machine you are logging in to. Older versions of Remote Desktop were especially susceptible to this type of "man in the middle" attack. The work you are doing for this course is relatively benign and low risk, so you can click Connect.

    If you are using a computer at work, it's possible that Remote Desktop connections to machines outside your corporate firewall are blocked. If this is the case, you need to work with your IT administrator to open communication through port 3389 on your machine to all machines in the Amazon subnet. If you work in a high-security environment (or any environment with lots of red tape), getting approval to change a firewall rule like this may be difficult or impossible, and it will be easier to perform these steps from home instead.
  5. In the Password input box, carefully type or paste the decrypted administrator password that you obtained from your instructor (sometimes Windows will not allow you to paste a password). Then click OK.

    You may see a window warning you that the identity of the remote computer cannot be verified. You can ignore this warning and click Yes.

    In a few seconds, you should see Windows appear. You are now working in a remote desktop session that is connected to your Amazon EC2 instance. This behaves just like any program in Windows. You can minimize it or close it, but note that closing your remote desktop session does not stop your instance. Your instance will continue to accrue charges until you right-click it in the AWS Management Console and click Stop. This is what you should do if you are interrupted while performing these instructions, or need to take a break at any time.

    The first thing you'll do on your instance is change the administrator password to something easier to remember.
  6. On your instance (not your own computer), click Start > Administrative Tools > Computer Management.
  7. Expand Local Users and Groups and click Users.
  8. In the list of users, right-click Administrator and click Set Password > Proceed. Type and confirm a new password that you can remember. In the future, you can use this password when logging in to your instance.

You're now ready to begin working with your EC2 instance. You will probably have to do the above steps just one time during this course; however, if something goes wrong with your instance, you can terminate it and create a new instance. You will need to license the software on any new instance you create, repeating the steps above.

Typically when you log in to your instance, you'll open Remote Desktop Connection and type the user name Administrator, followed by the new password that you set above. To end your session, you can just close the remote desktop window. If you are going away for more than an hour, also make sure to stop your instance in the AWS Management Console.

Note that there is an easier way to access your instance: From the AWS console with your instances showing, right click on your instance and then download a shortcut to your instance, which is an rdp file. The disadvantage of this method is that you will not have access to your local hard drive contents. Still, it's a handy option if you don't need that local access. 

Fortunately, you don't have to repeat all the previous steps to complete the Launch Instance wizard every time you want to use Amazon EC2. Once you have an instance created, it's fairly easy to log in, start, and and stop it. Before we talk about logging in, let's cover the basics of how to stop and start the instance. You'll need to begin using these techniques immediately, every time you use your instance, in order to keep costs down.

When people begin using Amazon EC2, they often ask about the difference between logging out, stopping, or terminating an instance.

  • You can close your Windows Remote Desktop Connection session or click the Windows Log Out button when you are finished using your instance. However, this does not stop the instance and you continue to accrue charges for it.
  • You can stop the instance, which is akin to pressing the power button to turn off your physical machine on your desktop: the machine is still there, and you can start it later, but it's not using any resources like electricity, spinning its CPU, etc., and it's not getting charged by Amazon. (Amazon does continue to charge you for the disk space your instance is using, but this is a relatively small charge.)  When you are working through this course, you should stop the instance when you are finished working on the lessons for the day. When you are ready to go back to the lessons, you can start the instance and continue working with your programs and data.
  • You can terminate an instance, which makes the instance go away forever. The only thing left behind is any disk drive that was attached to the instance. Terminating your instance will hopefully not be necessary in this course; however, you can keep it as an option if your instance gets corrupted. If you terminate your instance, then you will have to create a new one using the steps in the previous section of the lesson for the Launch Instance wizard.

If you fail to stop your instances after you have finished working, you will quickly use up the Amazon education credits that you have been allotted for this course. You are responsible for covering the cost overrun with your own money.

Below are some reference instructions that you can use to stop and start your instance (Do not stop your instance for at least 20 minutes after you first launch it. It needs time to configure Windows and OpenGeo Server for the first time.)

You can return to this page throughout the course if you need help remembering how to stop and start your instance. Be aware that a few of the instructions apply only to instances running ArcGIS Server, which you will use in Lesson 2. These steps are clearly marked as being ArcGIS Server-specific.

Stopping your GeoServer instance

  1. Log in to the AWS Management Console and click the Amazon EC2 tab.
  2. Click Instances.
  3. Right-click your instance and click Stop.

This stops the clock on the charges for running your instance. When an instance is stopped, no one can use your server and you cannot log in.

Starting a GeoServer instance and re-associating the Elastic IP

When you start your instance, it takes a few minutes to boot up, but you shouldn't have to wait the full 20 minutes that you waited when you first launched the instance. Whenever you start an instance, you also have to remember to re-associate the Elastic IP. Always follow the instructions below when you start your instance:

  1. Log in to the AWS Management Console and click the Amazon EC2 tab.
  2. Click Instances.
  3. Right-click your instance and click Start.
  4. Wait at least 10 minutes so that the machine can start up and configure itself correctly. 

    All instances require a few minutes to start up. ArcGIS Server instances especially require this time in order to run scripts that sync up the new machine name with ArcGIS Server. If these get out of sync, it can be difficult to fix and may require you to create a new instance.
  5. Click Elastic IPs and check the box next to your Elastic IP.
  6. Click Associate Address, choose your instance from the drop-down list, and click Yes, Associate.

After a few minutes your instance will be ready to use with its Elastic IP. After enough times of repeating this action you should have these instructions memorized.

Stopping and starting ArcGIS Server instances

When you start using ArcGIS Server on Amazon EC2, you'll launch, stop, and start your site using an application called Cloud Builder. You should not use the above instructions to stop and start an ArcGIS Server instance that you have launched with Cloud Builder. Instead, use the Start and Stop buttons that are included in Cloud Builder. These start and stop your instances behind the scenes, but they also perform other actions that are essential to stop and start your ArcGIS Server site smoothly.

Viewing your bill

To view your accrued charges at any time, go to http://aws.amazon.com/account and click Account Activity. Log in and you'll be able to see a detailed breakdown of all the charges you've incurred on Amazon EC2.

I recommend that you view your bill after every lesson so that you can understand whether you are in danger of using up all your education credits. If you consistently stop your instances after you are finished working, it is unlikely that you will reach the limit. However, if you are close to reaching the limit or have already surpassed it, please inform an instructor to discuss your options.

Setting up a cloud compute instance


Let's create an EC2 instance that is running Windows Server. You can use this either as a server or a workstation. Before you attempt this part of the lesson, you need to make sure you've done the following things. You need to have:
  • Obtained an Amazon account and enabled it for use with Amazon EC2
  • Applied the Amazon education credits to your account

If you have any doubt about one of these items, contact the course instructor by posting below or by email.

Here are the steps for getting a server running on Amazon EC2. Since Amazon can potentially update their site at any given time, some minor adjustments may be required for these steps. Contact the instructor if you have questions, or, if you find an issue that you are able to work around, please mention it in a comment below.

  1. Open a web browser to http://aws.amazon.com/console

    This application is called the AWS Management Console, and it helps you create and manage things on EC2, such as instances. This app has some quirks, and I've found that I have to run it in the Google Chrome browser to completely avoid them. Sometimes it will work in Firefox.
  2. Click Sign in to the AWS Management Console.
  3. Type your e-mail address (which is your Amazon account name) and password, then click Sign in using our secure server
    You should be taken to a screen with a bunch of tabs across the top, such as Elastic Beanstalk, S3, etc. These represent all the types of web services that Amazon offers. For now, you're interested only in EC2, which is Amazon's set of web services for renting hardware infrastructure.
  4. Click the Amazon EC2 tab. On the right, you'll see a summary of all the items you have running in Amazon EC2. There should be nothing listed. On the left, you'll see a menu of different categories of things you can create in EC2, such as Instances, Volumes, Elastic IPs, etc. You'll learn about a few of these as we go along.

    One thing you can select at this time is the region you want to work in. Amazon runs EC2 from various data centers placed around the world. You can choose which data center, or region, will house the resources you create. Typically, the closer you can place your region to your end users, the faster your services and apps will appear. But some organizations may also pick a region based on legal requirements relating to countries that can or cannot house their data. If you are working on this course from a continent other than North America, you may want to change the Region dropdown to choose a data center closer to you. Be aware that costs are slightly higher in some regions. You can see a list of costs at http://aws.amazon.com/pricing/ec2.
  5. From the left menu, click Instances. Ensure that you are using the "US East" region (there's a drop-down list in the upper left), then click the Launch Instance button. 

    In the window that appears, titled Create a new instance, be certain the radio button for Launch Classic Wizard is selected. Then hit Continue. A wizard appears that will help you create an instance. The first thing you're going to do is choose the Amazon Machine Image (AMI) that will determine the software and settings on your instance. You'll see that Amazon offers some basic AMIs containing just an operating system. Select one of the stock Windows instances, without a database installed.  
  6. You are taken to the Instance Details panel. Here you can choose the availability zone (AZ) in which your instance will run. AZs are Amazon's way of isolating machines within a region to minimize the chance of your site going down. For example, the US West region contains three AZs. The AZs are located in physically separate facilities that use different power grids, are built on different flood plains, etc. If your site contains multiple servers, you can place them in different AZs to minimize the chance of the full loss of your site if a data center is damaged. In your case, where you choosing one instance, you'll just leave this as No Preference to let Amazon select an available zone.
    On this panel you'll also choose the size, or computing power, of your instance. The instance size that you choose drastically effects the price that you pay, so follow these instructions carefully.
  7. Click the Instance Type dropdown and select High-CPU Medium (c1.medium). This is the largest type of instance for a 32-bit Windows server. It has good performance, and is economical (29 cents an hour for Windows, 17 cents an hour for Linux). Then click Continue.

    You're now viewing a panel where you can choose even more instance details. Some of these are beyond the scope of this course. However, you will enable Termination Protection. Terminating your instance deletes it forever. Termination Protection is nothing fancy; it just prevents you from terminating an instance until you explicitly disable termination protection on the instance. It's a way of making you go through an extra step to make sure you don't accidentally do something you didn't intend to do, which is helpful for beginners.
  8. Check the Termination Protection checkbox, then click Continue

    Now you're at a place where you can type a name for your instance. It used to be that your instances in the console were just assigned an ID. This was hard to keep track of once you had more than just a few instances, so Amazon allows you to type other metadata about the instance. This is stored as name/value pairs.
  9. In the Value column, go to the first text box (the one right across from Name), and type a name for your instance, such as Course Instance. Then click Continue.

    This first time you log in to your instance, you'll need to either know a pre-set Administrator password, or have one generated using a special file called a key pair in order to retrieve it. On this panel of the wizard you can create a key pair file.
  10. Click Create a new Key Pair, type a name for your key pair, then click Create & Download your Key Pair. A small, text-based file with the extension of .pem will be downloaded to your machine. Keep this key pair file in a safe place that you remember. You'll use it later in the course, although not with this OpenGeo instance.

    After downloading your key pair, you may be automatically taken to the next page of the wizard, but if not, click Continue. Now you will set some rules about what type of incoming Internet traffic can access your server. Amazon provides a firewall around every new instance that blocks all incoming traffic. You have to selectively "poke holes" in this firewall to allow appropriate types of communication with your server. In EC2 lingo, the set of rules that you create is called a Security Group.
  11. Click Create a new Security Group and type a group name and description.
  12. Find the Create a new rule drop-down list and choose HTTP. Then click Add Rule. You have just allowed HTTP access on Port 80 to everyone, thereby letting Internet users access your web services. Port 80 is the most common port used on the Internet for incoming web traffic into a server.
  13. From the Create a new rule drop-down list, choose RDP. Then click Add Rule. This rule is necessary so you can log in to and administer your instance, since Windows Remote Desktop requires port 3389 to be open.

    You may have noticed when you added those rules that you could specify a Source IP address using notation such as 0.0.0.0/0. This is called classless inter-domain routing (CIDR) notation and allows you to specify an IP address or a range of addresses that are allowed to connect through the port. With EC2 instances that you wanted to keep very secure, you would not typically open RDP access to all addresses (which we allowed by using 0.0.0.0/0). Instead, you would specify your IP address or your organization's range of IP addresses using CIDR notation. 

    If you know your IP address (easily attainable by visiting a site like whatismyip.com), try applying it to this rule to lock down your instance even more. Just be sure to put /32 at the end (for example: 67.168.99.22/32). The /32 is CIDR's way of limiting access to just the one address specified, instead of a range of addresses.

    Once you're done applying these two rules for HTTP and RDP, click Continue. EC2 now has all the information it has to launch your instance at this point.
  14. The wizard window now shows a summary of the instance that will be created. Examine it. Follow the Edit Firewall link and be certain that your Security Group is the one being used, then click Continue to return to the summary view. ClickLaunch. You'll see a message telling you that the instance is launching, which you can close.

    At this moment, somewhere in Amazon's data center, a virtual machine is being created for you with Windows on it.
  15. As the launch process proceeds, the My Instances window should appear, and you will see your instance listed. (You can also select the Instances link from the left menu of the AWS Management Console.). Within a minute or two, you'll see its status change from pending to running, but this does not mean the instance is ready yet. It takes around 20 minutes for Windows and the software running on your instance to configure itself. It's best not to disturb the instance while this is occurring.

    Every instance you create has a public-facing address, or Public DNS, that can be used to reference the instance from anywhere on the Internet. The challenge is that this address changes every time your stop and then start your instance. To give your machine a more permanent address, you'll set up an Amazon Elastic IP. This is an unchanging address that Amazon allocates to you for your use. You can then associate it with any instance you choose. Every time you stop and start the instance, you'll associate it with this IP address.
  16. At least 10 minutes after performing the previous step, open the AWS Management Console and click Elastic IPs.
  17. Click Allocate New Address, leave the drop-down list as EC2, and click Yes, Allocate.

    You should see an address appear in your list of Elastic IPs, such as 107.20.220.152.
  18. Click Associate Address, choose your instance from the drop-down (an instance must be running to appear here) and click Yes, Associate.

Once you launch an instance, the instance starts automatically and your Amazon bill begins accruing. It's very important to understand that you begin amassing charges right away; Amazon does not wait until you log in to your instance to begin charging you. In order to control costs, you need to stop your instance whenever you aren't using it. Before you take a break, please immediately continue reading the next section of the lesson to understand how to properly stop and start your instance.

Search This Blog

Full Text  Tag

Tag Cloud

Categories

Subscribe