pwo: user name with odd unicode characters causing Mysql2::Error: Incorrect string value

Mysql2::Error: Incorrect string value: '\xEF\xBC\x8Cpen...' for column 'student_name' at row 1: INSERT INTO `instructor_quiz_scores` (`student_name`, `quiz_name`, `score`, `date_completed`, `student_id`, `instructor_id`, `quiz_id`, `created_at`, `updated_at`) VALUES ('wei，peng #27111', 'Weather service Quiz', -1.0, '2016-05-31 14:06:58', 26781, 18713, 439635, '2016-05-31 14:06:58', '2016-05-31 14:06:58')

Mysql2::Error: Incorrect string value: '\xEF\xBC\x8C Mi...' for column 'student_name' at row 1: INSERT INTO `instructor_quiz_scores` (`student_name`, `quiz_name`, `score`, `date_completed`, `student_id`, `instructor_id`, `quiz_id`, `created_at`, `updated_at`) VALUES ('Yang， Mingsai#27113', 'Weather service Quiz', -1.0, '2016-05-31 14:11:58', 26784, 18713, 439654, '2016-05-31 14:11:58', '2016-05-31 14:11:58')

2.2.3 :013 > User.find_by(email:'463874549@qq.com').name

 => "Yang， Mingsai#27113" 

2.2.3 :014 > User.find_by(email:'463874549@qq.com').name.length

 => 19 

2.2.3 :015 > User.find_by(email:'463874549@qq.com').name.bytes.length

 => 21 

2.2.3 :016 > User.find_by(email:'463874549@qq.com').name.ascii_only?

 => false 

root cause

weird unicode chars in user name, instructor_quiz_scores user name column can't deal with them

http://stackoverflow.com/questions/22464011/mysql2error-incorrect-string-value

to fix

find these entries in the prod database using Sequel Pro, delete the weirdo unicode comma.



Steve,
Can you investigate this?  We have two reports of this issue this morning.  Both are students from the same instructor ( Marcus Callahan: marcus.callahan@afa.edu ).

The other student account is:  463874549@qq.com

I found it interesting that when I log in as them, their student tools says there are no instructor quizzes so I can't tell what is occurring.  Based on the time stamp of the emails, whatever it is that is happening, is occurring this morning.


Mike Marquez | Technical Support
Aviation Supplies & Academics, Inc.
7005 132nd Place SE | Newcastle, Wa. 98059
800-272-2359 | 425-235-1500 | Visit us on the web: www.asa2fly.com



-----Original Message-----
From: Brian Snider [mailto:brian@asa2fly.com]
Sent: Tuesday, May 31, 2016 8:26 AM
To: michael@asa2fly.com
Subject: FW: Some errors about my account



Brian W. Snider, Marketing Coordinator
Aviation Supplies & Academics, Inc.
7005 132nd Place SE
Newcastle, WA 98059-3153
Phone 800-426-8338 or 425-235-1500
Fax 425-235-0128
Website www.asa2fly.com
Email brian@asa2fly.com

-----Original Message-----
From: PengWei@dcamail.net [mailto:PengWei@dcamail.net]
Sent: Tuesday, May 31, 2016 8:10 AM
To: webmaster@asa2fly.com
Subject: Some errors about my account

Hello,I am a student pilot in Aerosim ,and I am in ground lesson now .I followed an instructor already,but I can't do his quiz.everytime when I take the quiz,it shows "we are sorry, but something went wrong".so,can you help me to correct it ? Thank you very much.

export qti notes

##################################################
# core question unit is an item
##################################################
questions are (can be) html
wrapped in CDATA
presentation block: question and answer stems
resprocessing block:
  scoring and answer-specific feedback
  referes to the response_lid ident
  one respcondition block for each answer choice

some of the questions cause the yaml output for the entire file to get borked (e.g. asanum 8393.2):
            <mattext texttype="text/html"><![CDATA[See the Update PDF for the figure associated with this question. You must follow the Order of Operations rule:
1.(v31) + (v43) ÷ 172

<item title="The working voltage" ident="QUE_1004">
<presentation>
  <material>
  <mattext texttype="text/html"><![CDATA[The working voltage of a capacitor in an ac circuit should be]]></mattext>
  </material>
  <response_lid ident="QUE_1005_RL" rcardinality="Single" rtiming="No">
    <render_choice>
    <response_label ident="QUE_1006_A1">
    <material>
    <mattext texttype="text/html"><![CDATA[equal to the highest applied voltage.]]></mattext>
    </material>
    </response_label>

    <response_label ident="QUE_1007_A2">
    <material>
    <mattext texttype="text/html"><![CDATA[at least 20 percent greater than the highest applied voltage.]]></mattext>
    </material>
    </response_label>

    <response_label ident="QUE_1008_A3">
    <material>
    <mattext texttype="text/html"><![CDATA[at least 50 percent greater than the highest applied voltage.]]></mattext>
    </material>
    </response_label>

    </render_choice>
  </response_lid>
</presentation>

<resprocessing>
  <outcomes>
    <decvar vartype="Integer" defaultval="0" varname="que_score"/>
  </outcomes>
  <respcondition>
    <conditionvar>
      <varequal respident="QUE_1005_RL">QUE_1006_A1</varequal>
    </conditionvar>
    <setvar varname="que_score" action="Add">0</setvar>
    <displayfeedback feedbacktype="Response" linkrefid="QUE_1003_ALL"/>
  </respcondition>
  <respcondition>
    <conditionvar>
      <varequal respident="QUE_1005_RL">QUE_1007_A2</varequal>
    </conditionvar>
    <setvar varname="que_score" action="Add">0</setvar>
    <displayfeedback feedbacktype="Response" linkrefid="QUE_1003_ALL"/>
  </respcondition>
  <respcondition>
  <conditionvar>
    <varequal respident="QUE_1005_RL">QUE_1008_A3</varequal>
  </conditionvar>
  <setvar varname="que_score" action="Set">1</setvar>
  <displayfeedback feedbacktype="Response" linkrefid="QUE_1003_ALL"/>
  </respcondition>
  <respcondition>
  <conditionvar>
    <other/>
  </conditionvar>
  <displayfeedback feedbacktype="Response" linkrefid="QUE_1003_ALL"/>
  </respcondition>
</resprocessing>

<itemfeedback ident="QUE_1003_ALL" view="All">
  <material>
  <mattext texttype="text/html"><![CDATA[The working voltage of a capacitor is the highest voltage that can be steadily applied to it without the danger of the dielectric breaking down. The working voltage depends upon the material used as the dielectric and on its thickness. A capacitor used in an AC circuit should have a working voltage at least 50 percent greater than the highest voltage that will be applied to it.]]></mattext>
  </material>
</itemfeedback>
</item>

/Users/smr/current_projects/pws2016/app/controllers/sandbox_controller.rb

  def qti_export
    # quiz_id=40
    # @quiz = InstructorQuiz.find(quiz_id)
    # @qids = JSON.parse(@quiz.qids)
    # @qids_json = JSON.parse(@quiz.qids)

    @qids = (1..100)  #for testing

    # @qids = @qids[450, 1]  #slice for testing
    # @qids = @qids[0, 100]  #slice for testing

    qdbm = QuestionDatabaseMiner.new
    @questions = qdbm.questions_for_qids(@qids)

    incremental_id = 1000
    respident = ""
    ans1_ident = ""
    ans2_ident = ""
    ans3_ident = ""

    builder = Nokogiri::XML::Builder.new do |xml|
      xml.questestinterop {  # root node can be named whatever
        xml.assessment(title:"QUIZ NAME HERE", ident:"A#{incremental_id+=1}") {
          xml.section(title:"Main", ident:"S#{incremental_id+=1}") {
            (@questions).each_index { |i|
              q_obj = @questions[i]
              qnum = i + 1
              fdbk_link_id = "QUE_#{incremental_id+=1}_ALL"
              # xml.debug_data(index:i, asaqnum:q_obj.asaqnum)
              xml.item(title:"Question #{qnum}", ident:"QUE_#{incremental_id+=1}") {
                # presentation block
                xml.presentation() {
                  xml.material {
                    xml.mattext q_obj.question
                  }
                  respident = "QUE_#{incremental_id+=1}_RL"
                  xml.response_lid(ident:respident, rcardinality:"Single", rtiming:"No") {
                    xml.render_choice {
                      ans1_ident = "QUE_#{incremental_id+=1}_A1"
                      xml.response_label(ident:ans1_ident) {
                        xml.material {
                          xml.mattext(texttype:"text/html") { xml.cdata(q_obj.answerA.replace_prepware_placeholders_utf8!) }
                        }
                      }
                      ans2_ident = "QUE_#{incremental_id+=1}_A2"
                      xml.response_label(ident:ans2_ident) {
                        xml.material {
                          xml.mattext(texttype:"text/html") { xml.cdata(q_obj.answerB.replace_prepware_placeholders_utf8!) }
                        }
                      }
                      ans3_ident = "QUE_#{incremental_id+=1}_A3"
                      xml.response_label(ident:ans3_ident) {
                        xml.material {
                          xml.mattext(texttype:"text/html") { xml.cdata(q_obj.answerC.replace_prepware_placeholders_utf8!) }
                        }
                      }
                    }
                  }
                }
                # resprocessing block
                xml.resprocessing {
                  xml.outcomes {
                    xml.decvar(vartype:"Integer", defaultval:"0", varname:"que_score")
                  }
                  # answer A
                  xml.respcondition {
                    xml.conditionvar {
                      xml.varequal(respident:respident) { xml.text(ans1_ident) }
                    }
                    xml.setvar(varname:"que_score", action:"Set") { 
                      isCorrect = (q_obj.correctChoice == "A") ? 1 : 0
                      xml.text(isCorrect) 
                    }
                    xml.displayfeedback(feedbacktype:"Response", linkrefid:fdbk_link_id)
                  }
                  # answer B
                  xml.respcondition {
                    xml.conditionvar {
                      xml.varequal(respident:respident) { xml.text(ans2_ident) }
                    }
                    xml.setvar(varname:"que_score", action:"Set") { 
                      isCorrect = (q_obj.correctChoice == "B") ? 1 : 0
                      xml.text(isCorrect) 
                    }
                    xml.displayfeedback(feedbacktype:"Response", linkrefid:fdbk_link_id)
                  }
                  # answer C
                  xml.respcondition {
                    xml.conditionvar {
                      xml.varequal(respident:respident) { xml.text(ans3_ident) }
                    }
                    xml.setvar(varname:"que_score", action:"Set") { 
                      isCorrect = (q_obj.correctChoice == "C") ? 1 : 0
                      xml.text(isCorrect) 
                    }
                    xml.displayfeedback(feedbacktype:"Response", linkrefid:fdbk_link_id)
                  }
                }
                # itemfeedback block
                xml.itemfeedback(ident:fdbk_link_id, view:"All") {
                  xml.material {
                    xml.mattext(texttype:"text/html") { 
                      xml.cdata(q_obj.explanationCommonToAllAnswerStems.replace_prepware_placeholders_utf8!) 
                    }
                  }
                }
              }
            }
          }
        }
      }
    end
    @xml = builder.to_xml
    aFile = File.new("/tmp/qti_test.xml", "w")
    aFile.write(@xml)
    aFile.close
  end

Upgrade-the-hard-drive-on-a-MacBook-Pro-HDD-SSD

http://www.instructables.com/id/Upgrade-the-hard-drive-on-a-MacBook-Pro-HDD-SSD/?ALLSTEPS

rake task to tweak view_in_demo_mode flag; unlock additional content for the grs Demo

/Users/smr/current_projects/pws2016/lib/tasks/modify_grs_courses.rake

namespace :db do
  
  desc "set view_in_demo_mode flag for grs-ins courses"
  task smr_set_view_in_demo_mode_flag: :environment do 

    GrsCourse.order(created_at: :desc).each do |course|
      if course.identifier == 'ins'
        course.stages.first.lessons.first.requirements.each { |r| 
          r.update_attribute(:view_in_demo_mode, true)
        }
      end

    end

  end
end

this approach is fairly slow, might be easier just to do it with sql

tweak opf file in The Wise Mans Fear.epub to circumvent an import error in iBooks

The Wise Mans Fear.epub tweak

try to import inyo ibooks

error:

Line 4: Namespace prefix opf for file-as on creator is not defined

http://www.mobileread.com/forums/showthread.php?t=222578

/Users/smr/Desktop/scratch/rothfuss/

/Users/smr/Desktop/scratch/rothfuss/The Wise Mans Fear.epub

duplicate and rename to a zip:

/Users/smr/Desktop/scratch/rothfuss/The Wise Mans Fear.zip

make a folder to contain it (it unzips in-place as a few folders ):

/Users/smr/Desktop/scratch/rothfuss/expanded/

copy the renamed zip over:

/Users/smr/Desktop/scratch/rothfuss/expanded/The Wise Mans Fear.zip

expand in a terminal window using unzip

tweak file:

/Users/smr/Desktop/scratch/rothfuss/expanded/OEBPS/roth_9781101486405_oeb_opf_r1.opf

error is in the metadata block:

  <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">

      <dc:title>The Wise Man's Fear</dc:title>

      <dc:creator opf:file-as="Rothfuss, Patrick">Patrick Rothfuss</dc:creator>

      <dc:publisher>PENGUIN group</dc:publisher>

      <dc:format/>

      <dc:date>2011-01-25</dc:date>

      <dc:subject/>

      <dc:description/>

      <dc:rights>Copyright © 2011 by Patrick Rothfuss</dc:rights>

      <dc:identifier id="bookid">9781101486405</dc:identifier>

      <dc:language>en</dc:language>

      <meta name="cover" content="coverimagestandard"/>

  </metadata>

as a fix, i just deleted the offending line.

select all

compress 4 items

this creates Archive.zip

rename Archive.zip to 

The Wise Mans Fear.epub

research canvas and QTI

1. research canvas and QTI
1. canvas: You can easily import quizzes from programs that create QTI files. If a program does not create QTI files, it cannot be imported into Canvas.
2. http://www.imsglobal.org/question/qtiv1p2/imsqti_litev1p2.html
3. http://www.imsglobal.org/question/index.html
4. http://guides.instructure.com/s/2204/m/4152/l/40724-how-do-i-import-quizzes-from-qti-packages
5. http://www.xquestion.com/Examples.html

same QTI file: XQuestionSample.zip

ruby script to create a catalog of sublime snippets by dissecting xml files

#!/usr/bin/env ruby
# coding: utf-8

require "nokogiri"
require 'find'

hits = []

path_to_snippets_folder = "/Users/smr/Library/Application Support/Sublime Text 2/Packages/User/_my-snippets/"

snippet_file_paths = []
Find.find(path_to_snippets_folder) do |path|
  snippet_file_paths << path if path =~ /.*\.sublime-snippet/
end

snippet_file_paths.each do |path|

  filename = File.basename(path, ".*")
  puts "***processing filename=#{filename}"

  f = File.open(path)
  doc = Nokogiri::XML(f)
  doc.xpath('snippet').each do |snippet|
    description_node = snippet.at_xpath('description')
    description = ""
    if description_node
      description =  description_node.text
    end

    tabTrigger_node = snippet.at_xpath('tabTrigger')
    tabTrigger = ""
    if tabTrigger_node
      tabTrigger =  tabTrigger_node.text
    end

    content_node = snippet.at_xpath('content')
    content = ""
    if content_node
      content =  content_node.text
    end

    hits.push( { tabTrigger:tabTrigger, description:description, content:content } )
  end
  f.close
end


# create an html file
builder = Nokogiri::HTML::Builder.new(:encoding => 'UTF-8') do |doc|
doc.html {
  doc.head {
    doc.link(href: "http://netdna.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css", rel: "stylesheet")
    doc.script(src:"https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js")
  }
  doc.body {
    doc.div(class: "container") {
      doc.h2("sublime snippets")
      # sort an array of dict by one of the values of a dict
      hits.sort_by {|d| d[:tabTrigger]}.each { |dict| 
        doc.div {
          doc.h3(dict[:tabTrigger])
          doc.p(dict[:description]) if dict[:description].empty? == false
          doc.pre.prettyprint(dict[:content])
          doc.hr
        }
      }
    }
  }
}
end


path = "/tmp/sublime-snippets-snippets.html"
aFile = File.new(path, "w")
aFile.write(builder.to_html)
aFile.close

system "open", path

ruby script to create a catalog of xcode snippets by dissecting plist files

#!/usr/bin/env ruby
# coding: utf-8

require 'fileutils'
require "nokogiri"


hits = []

path_to_snippets_folder = "/Users/smr/Library/Developer/Xcode/UserData/CodeSnippets/"
FileUtils.cd(path_to_snippets_folder)
Dir.glob("*.{codesnippet}").each { |file|
  f = File.open(file)
  doc = Nokogiri::XML(f)
  f.close

  shortcut = ""
  doc.xpath('//key[text()="IDECodeSnippetCompletionPrefix"]').each { |key|
    shortcut = key.next_element().text()
  }

  summary = ""
  doc.xpath('//key[text()="IDECodeSnippetSummary"]').each { |key|
    summary = key.next_element().text()
  }

  snippet = ""
  doc.xpath('//key[text()="IDECodeSnippetContents"]').each { |key|
    snippet = key.next_element().text()
  }

  hits.push( { shortcut:shortcut, summary:summary, snippet:snippet } )
}

# create an html file
builder = Nokogiri::HTML::Builder.new(:encoding => 'UTF-8') do |doc|
doc.html {
  doc.head {
    doc.link(href: "http://netdna.bootstrapcdn.com/bootstrap/3.0.2/css/bootstrap.min.css", rel: "stylesheet")
    doc.script(src:"https://google-code-prettify.googlecode.com/svn/loader/run_prettify.js")
  }
  doc.body {
    doc.div(class: "container") {
      doc.h2("xcode snippets")
      # sort an array of dict by one of the values of a dict
      hits.sort_by {|d| d[:shortcut]}.each { |dict| 
        doc.div {
          doc.h3(dict[:shortcut])
          doc.p(dict[:summary]) if dict[:summary].empty? == false
          doc.pre.prettyprint(dict[:snippet])
          doc.hr
        }
      }
    }
  }
}
end


path = "/tmp/xcode-snippets.html"
aFile = File.new(path, "w")
aFile.write(builder.to_html)
aFile.close

system "open", path

respondus export tweaks

1. respondus import issue
1. http://util.prepwareschool.com/jira/browse/PWO-200
2. ✓ example file from wmu is the general database, all questions exported
3. ✓ multi-line questions are ok unless the lines begin with things like '1.', which confuses the respondus import parser.
4. ✓ '1-' seems to work
5. ✓ need to rework multi-line questions, replacing 1. with 1-
6. title?
7. https://www.respondus.com/products/respondus/guides.shtml
8. https://www.ucmo.edu/centralnet/Respondus/Creating_Test_Imports_Respondus.pdf
9. 1. https://wiki.csuchico.edu/confluence/display/tlpkb/Formatting+guide+for+importing+different+question+types+into+Respondus
   
   

quizmaker one-step redesign

1. current PWO issue
1. http://util.prepwareschool.com/jira/browse/PWO-91 QM one-step
1. ✓ review one step code in prepware mac version
2. ✓ remove testmap count and testmap% columns
3. review model backing non AR simple_form
1. ✓ validations
2. move logic from controller to model

generate release build email or FARAMT android using info from git log and build.gradle file

#!/usr/bin/env ruby
require 'fileutils'
require 'nokogiri'
require 'git'

git_repo_working_dir = '/Users/smr/current_projects/faramt-android/'


##################################################
# get sha of last commit from the git repo 
##################################################
g = Git.open(git_repo_working_dir)
commits = g.log(1)   # returns array of last N Git::Commit objects
last_commit = commits.first
sha_first_7 = last_commit.sha[0..6]

##################################################
# get the app version from the build.gradle 
##################################################
FileUtils.cd("/Users/smr/current_projects/faramt-android/faramt/")
filename = "build.gradle"
contents = File.read(filename)
aoa = contents.scan(/versionName "(.*)"/)
version = ""
if aoa.length
  match = aoa[0]
  if match.length
    version = match[0]
  end
end
puts "version=#{version}"

# name looks like FARAMT-1.39-b08ba16.apk
ipa_filename = "FARAMT-#{version}-#{sha_first_7}.apk"


##################################################
# generate email: mail body includes last 5 commits
##################################################
email_subject = "faramt for android #{version} (#{sha_first_7})"
mail_body = <<EOF
a new faramt for android build (version #{version}, build #{sha_first_7}) was posted to the asa ftp apps server at:
http://www.prepware.com/faramt_android/index.html
and is ready for testing.

EOF

commits = g.log(5)
commits.each do |commit|
  mail_body += "
commit #{commit.sha}
Author: #{commit.author.name} <#{commit.author.email}>
Date: #{commit.date.strftime('%A, %B %d, %Y')}

#{commit.message}

"
end

# escape double quotes
mail_body.gsub!(/"/, '\"')


##################################################
# use osacscript compose an email 
##################################################
# https://gist.github.com/dinge/6983008
# calling applescript from ruby
def osascript(script)
  system 'osascript', *script.split(/\n/).map { |line| ['-e', line] }.flatten
end


osascript <<EOF

set theSender to "stephen rouse <stephenr@asa2fly.com>"
set recipientAddressList to ["jackie@asa2fly.com", "sarah@asa2fly.com", "michael@asa2fly.com"]
set theSubject to "#{email_subject}"
set theContent to "#{mail_body}"
set theSignatureName to "asa2fly"

tell application "Mail"
    set msg to make new outgoing message with properties {subject:theSubject, sender:theSender, content:theContent, visible:true}
    set message signature of msg to signature theSignatureName
    tell msg
      repeat with i from 1 to count recipientAddressList
          make new to recipient at end of to recipients with properties {address:item i of recipientAddressList}
      end repeat
    end tell
end tell

return -- prevents "missing value" being returned by the script
EOF