cp11_old_14_WebIntegration

The Internet, or the Web, has evolved from some separate world into something that is everywhere and in everything. It has become a technology platform enabling a multitude许多 of different use cases. From a finance perspective, the following seem particularly noteworthy值得注意的:

Data provision供应/gathering
Web technology allows the provision of data and the gathering thereof in a simplified manner and generally at reduced costs; it also speeds up in general all associated processes. Large financial data providers, like Bloomberg and Thomson Reuters, rely
heavily on the Web and related technologies to provide the financial world with data in real time.

Trading/buying/selling
Using the Web also facilitates促进 trading of financial securities; even private investors today have access to professional trading facilities (e.g., online brokers like Interactive Brokers) and can trade securities in real time.

Application providing
Models like Software-as-a-Service (SaaS) allow both small companies, like startups, and large ones to provide applications in an efficient manner; even the smallest outfit组织 can today reach a global target audience at very little cost. Large corporations benefit  from web technologies, for example, when they use them to provide internal applications that are accessible and usable via any standard web browser, instead of installing such applications on hundreds or even thousands of different machines.

Communication
Of course, the Web facilitates communication within organizations and across organizations; the majority of today’s business and financial communication has moved from paper to the Web.

Commoditization商品化/scalability可伸缩性
Recent web technologies also allow for better virtualization, making web servers and servers in general a commodity that everybody can rent at rather low variable costs and that is easily scalable when requirements change; computing power and storage capacity become more and more comparable to electricity越来越可与电力媲美, which we are all used to getting from the plug sockets at home.

 

Web protocols
The first section shows how to transfer files via FTP and how to access websites via HTTP.

Web plotting
Web technologies generally allow for better interactivity and for better real-time support than standard approaches, for example, for plotting data; the second section introduces the plotting library Bokeh to generate interactive web plots and to realize
real-time plotting of financial data.

Web applications
One of Python’s strengths is its powerful web frameworks to develop web-based applications; one that is really Pythonic(python
风格) and that has become quite popular recently is Flask. Here illustrates techniques for developing web-based applications
using this framework.

Web services
Web services have become an important aspect of web-enabled applications; the last section shows how to develop a simple web service for the valuation of European options on the VSTOXX volatility index波动率指数.

ftplib

The File Transfer Protocol (FTP) is, as the name suggests, a protocol to transfer files over the Web.] Python provides a dedicated library to work with FTP called ftplib. https://docs.python.org/2/library/ftplib.html

httplib

Another important protocol, if not the most important one on the Web, is the HyperText Transfer Protocol (HTTP) .This protocol is used whenever a (HTML-based) web page is displayed in the browser. The Python library to work with HTTP is called httplib:https://docs.python.org/2/library/httplib.html

import http
import http.client

http = http.client.HTTPConnection('hilpish.com') #first need a connection to the HTTP server
http.request('GET','/index.htm') #send requests to ask for the index.htm page (file)
resp = http.getresponse() #use the getresponse method To test whether the connection was successful
resp.status, resp.reason #The returned object provides status information

content = resp.read() # first 100 characters of the file
content[:200]

index = content.find('E')
index

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-37-2f73890a9af2> in <module>
----> 1 index = content.find('E')
      2 index

TypeError: a bytes-like object is required, not 'str'

1st-Solution:

index = content.find(bytes('e', encoding='utf8')) #find the position/index of the first character 'e'
index

2nd-solution:

strContent=str(content, encoding='utf8') #find the position/index of the first character 'e'
strContent

number = strContent.count('e')
number

strContent[index:index + 29] #inspect the subsequent characters of the object 'e':

 

http.close()#Once you are finished, you should again close the connection to the server

urllib

There is another Python library that supports the use of different web protocols. It is called urllib.https://docs.python.org/2/library/urllib.html There is also a related library called urllib2. https://docs.python.org/2/library/urllib2.htmlBoth libraries are designed to work with arbitrary web resources, in the spirit of the “uniform” in URL (uniform resource locator).A standard use case, for example, is to retrieve files, like CSV data files, via the
Web. Begin by importing urllib:

The application of the library’s functions resembles that of both ftplib and httplib. Of course, we need a URL representing the web resource of interest (HTTP or FTP server, in general). For this example, we use the URL of Yahoo! Finance to retrieve stock price information in CSV format:

import urllib
url= 'http://hilpisch.com/tr_eikon_eod_data.csv'
connect = urllib.request.urlopen(url) #establish a connection to the resource
data = connect.read()
print(data)

path = '/Users/LlQ/pythonForFinance/cp11_old_14_WebIntegration/'
urllib.request.urlretrieve(url, path+'eod_data.csv')

csv = open(path + 'eod_data.csv','r')
csv.readlines()[:5]

csv.close()

Web Plotting

This section starts with generating static plots, then proceeds to interactive plots to finally arrive at real-time plotting.

Static Plots
First, a brief benchmark example using the pandas library based on a financial time series as used in the previous section:

import numpy as np
import pandas as pd
%matplotlib inline

url = 'http://hilpisch.com/tr_eikon_eod_data.csv'
data = pd.read_csv(url, parse_dates=['Date'])

data.plot(x='Date', y='MSFT.O', 
          figsize=(10,6), 
          title='Historical stock prices for Microsoft since January 2010',
          grid=True)
# tag: microsoft

Interactive Plots

REMARK: The original version used Bokeh for Web plotting. Plotly seems to be the more easy and intuitive way for generating interactive D3.js Web plots.

import pandas as pd
import cufflinks as cf

from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
init_notebook_mode(connected=True)

iplot(data.set_index('Date')['MSFT.O'].iplot(asFigure=True))

Templating

Basically, templating with Flask (Jinja2) works similarly to simple string replacements in Python: you have a basic string indicating where to replace what and some data to be inserted into the string object. Consider the following examples:

'%d, %d, %d' % (1,2,3)

'{}, {}, {}'.format(1,2,3)

'{}, {}, {}'.format(*'123')

Templating to generate HTML pages works pretty similarly. The major difference is that the string object “resembles” an HTML document (or a part thereof) and has commands for replacements and also, for example, ways of controlling the flow when rendering the template (e.g., the for loop). Missing information is added during the rendering procedure, as we added the integers to the string object in the previous examples. Consider now the following string object, containing partly standard HTML code and some template-specific code:

templt = '''<!doctype html>
    Just print out <b>numbers</b> provided to the template.
    <br><br>
    {% for number in numbers %}
        {{ number }}
    {% endfor %}
'''

So far, this is a string object only. We have to generate a Jinja2 Template object out of it before proceeding:

from jinja2 import Template
t=Template(templt)

This Template object has a method called render to make valid HTML code out of the template and some input values — in this case, some numbers via the parameter numbers:

html = t.render(numbers = range(5))
html

Such an object containing HTML code can be rendered in IPython Notebook as follows:

from IPython.display import HTML
HTML(html)

Of course, templating involves much more than this simple example can illustrate (e.g., inheritance). More details can be found at http://jinja.pocoo.org. However, the templates for the Tradechat application already include a number of important aspects. Specifically, we need the following templates:

layout.html
                  Defines the basic layout from which the other templates inherit
register.html
                  The template for the user registration page
login.html
                  The corresponding template for the user login
show_entries.html
                   The main page showing the comments in the chat room and, if the user is logged in, the text field for writing and                         posting comments

https://github.com/yhilpisch/py4fi/tree/master/python36/tradechat/templates

######### layout.html ##########

<!doctype html>
<title>Tradechat</title>
<link rel=stylesheet type=text/css
     href="{{ url_for('static', filename='style.css') }}">
<div class=page>
  <h1>Tradechat</h1>
  <div class=metanav>
  {% if not session.logged_in %}
    <a href="{{ url_for('login') }}">log in</a><br>
    <a href="{{ url_for('register') }}">register</a>
  {% else %}
    <a href="{{ url_for('logout') }}">log out</a>
  {% endif %}
  </div>
  {% for message in get_flashed_messages() %}
    <div class=flash>{{ message }}</div>
  {% endfor %}
  {% block body %}{% endblock %}
</div>

######### register.html ##########

{% extends "layout.html" %}
{% block body %}
  <h2>Register</h2>
  {% if error %}<p class=error><strong>Error:</strong> {{ error }}{% endif %}
  <form action="{{ url_for('register') }}" method=post>
    <dl>
      <dd><font size="-1">Username</font>
      <dd><input type=text name=username>
      <dd><font size="-1">Password</font>
      <dd><input type=password name=password>
      <dd><input type=submit value=Register>
    </dl>
  </form>
{% endblock %}

######### login.html ##########

{% extends "layout.html" %}
{% block body %}
  <h2>Login</h2>
  {% if error %}<p class=error><strong>Error:</strong> {{ error }}{% endif %}
  <form action="{{ url_for('login') }}" method=post>
    <dl>
      <dd><font size="-1">Username</font>
      <dd><input type=text name=username>
      <dd><font size="-1">Password</font>
      <dd><input type=password name=password>
      <dd><input type=submit value=Login>
    </dl>
  </form>
{% endblock %}

######### show_entries.html ##########

{% extends "layout.html" %}
{% block body %}
  {% if session.logged_in %}
    <form action="{{ url_for('add_entry') }}" method=post class=add-comment>
      <dl>
        <dd>What's up?
        <dd><textarea name=text rows=3 cols=40></textarea>
        <dd><input type=submit value=Post>
      </dl>
    </form>
  {% endif %}
  <ul class=comments>
  {% for comment in comments %}
    <li>{{ comment.comment|safe }}
         <font size="-2">({{ comment.user }} @ {{ comment.time }})</font>
  {% else %}
    <li><em>No comments so far.</em>
  {% endfor %}
  </ul>
{% endblock %}