BigQuery ML does a good job of hot-encoding strings, but it doesn’t handle arrays as I wish it did (stay tuned). ) One-hot encoding Now get ready for some SQL magic. WHERE tag1 IN (SELECT tag FROM active_tags)ĪND tag2 IN (SELECT tag FROM active_tags) SELECT *, MAX(questions) OVER(PARTITION BY tag1) questions_tag1įROM data, UNNEST(SPLIT(tags, '|')) tag1, UNNEST(SPLIT(tags, '|')) tag2 SELECT *, questions/questions_tag1 percent CREATE OR REPLACE TABLE `deleting.stack_overflow_tag_co_ocurrence`įROM `fh-bigquery.stackoverflow_archive.201906_posts_questions`
#Comicrack ios ipa plus#
So I’ll take these relationships and I’ll save them on an auxiliary table - plus a percentage of how frequently a relationship happens for each tag. 提示：共现标签 Let’s find tags that usually go together:Ĭo-occurring tags on Stack Overflow questions Top Stack Overflow tags by number of questions. In this picture I only have 240 tags - how would you group and categorize 4,000+ of them? # Tags with >180 questions since 2018įROM `fh-bigquery.stackoverflow_archive.201906_posts_questions`, 4,000+ tags are a lot These are the most active Stack Overflow tags since 2018 - they’re a lot.
You can check out more about working with Stack Overflow data and BigQuery here and here. In this post he works with BigQuery – Google’s serverless data warehouse – to run k-means clustering over Stack Overflow’s published dataset, which is refreshed and uploaded to Google’s Cloud once a quarter. Visualizing a universe of clustered tags.įelipe Hoffa is a Developer Advocate for Google Cloud.
0 kommentar(er)
